A classic example of how you can nearly always improve performance by sacrificing something else, e.g. simplicity, or conformance to standards, or potential for change, or ease of writing an application to produce or consume the data.

Just don't do it unless (a) you know you need the extra performance, and (b) you know what you're sacrificing to get it.

Michael Kay

Saxonica

On 15 Nov 2017, at 12:07, Costello, Roger L. <costello@m...> wrote:

Hi Folks,

 

I recently read an article about Facebook dumping JSON due to inadequate performance of JSON parsers. See below. I wonder how XML parsers would stack up. Do you have numbers?

 

Parse a 20KB stream

Facebook requirement: must not exceed the UI frame refresh interval of 16.6 ms

JSON parser: 35 ms to parse a JSON stream of 20 KB (a typical response size for Facebook)

XML parser: ??

 

Parser initialization

Facebook requirement: Not specified

JSON parser: 100 ms to 200 ms before it can start parsing

XML parser: ??

 

Garbage collection

Facebook requirement: Not specified

JSON parser: 100 KB of transient memory allocated when parsing a JSON stream of 20 KB

XML parser: ??

 

Here’s why Facebook dumped JSON

As you know, people use Facebook to create connections with family and friends. Facebook stores all the data that makes up the social graph of these connections. On mobile clients, the entire graph can’t be downloaded, so only a node and some of its connections are downloaded as a local tree structure.

One of the key issues Facebook needed to address was how to represent and store the social graph data in an app. Their initial approach was to store the data in JSON. They used the Jackson JSON parser on Android. However, they dumped JSON after finding these issues:

Parsing speed. It took 35 ms to parse a JSON stream of 20 KB (a typical response size for Facebook), which exceeds the UI frame refresh interval of 16.6 ms. Users were unable to load stories on demand from disk cache without observing frame drops (visual stutters) while scrolling.

Parser initialization. A JSON parser needs to build a field mapping before it can start parsing, which can take 100 ms to 200 ms, substantially slowing down the application start time.

Garbage collection. A lot of small objects are created during JSON parsing, and testing revealed around 100 KB of transient memory was allocated when parsing a JSON stream of 20 KB, which placed significant pressure on the garbage collector.

Facebook needed a better storage format to increase the performance of their Android app.

What did they choose?

Answer: Google FlatBuffers.

https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/

/Roger