![]() To improve our Widget Shipping platform and support dropshipping our widgets, we've decided to implement Dynamic Forks. You can generate the benchmark data with benchmarks/gen.sh.Parsing and Formatting JSON with JQ Transform Task Order Fulfillment Codelab part 7 The benchmarks were run using hyperfine on a preheated (very loud) Macbook Pro 13 mid-2019 i7 2.8GHz 16GB 256GB. CommandĬat benchbig.json | jql '("countries" ((keys) ("name")))' > out.jsonĬat benchbig.json | jq '.countries.name' > out.jsonĬat benchmedium.json | jql '("countries" ((keys) ("name")))' > out.jsonĬat benchmedium.json | jq '.countries.name' > out.json The benchbig.json file contains 2^20 repetitions of the json we've been using in the examples so far, and the benchmedium.json file contains 20. What I will say though, is that jql is definitely not slow and you can freely cut your way through gigabytes of data quickly, as in the following benchmarks it was 2-3 times faster than jq. Especially since jq has various much more advanced functionalities. This is a fairly synthetic benchmark (it's the first that came to my mind), so I won't say jql is faster than jq. Ifte: (Expression x Expression x Expression) -> (Expression) ) -> (Expression]])Īnd,or: (Expression.) -> (Expression) With two args: (Expression x Expression) -> (Expression]) With one arg: (Expression) -> (Expression]) Join: (Expression.) -> (Expression)įilter: (Expression) -> (Expression)Įq,lt,gt: (Expression x Expression) -> (Expression) Sprintf: (Expression x Expression.) -> (Expression) Object: ((Expression x Expression).) -> (Expression) With two args: (Expression x Expression) -> (Expression)Īrray: (Expression.) -> (Expression]) (T's can vary) With one arg: (Expression) -> (Expression) = (elem position (id)) ![]() You've got eq, lt and gt, all working as you'd probably expect: > cat test.json | jql '("countries" ((keys) (sprintf "%s population: %.0f" ("name") (error "test message"))))'Ħ 00:17:04 error getting expression value for object: couldn't get transformed value for field countries with value map map]: couldn't get element using position at array index 0: couldn't get transformed value for index 0 with value map: couldn't evaluate sprintf argument with index 1: Message: test message This way we can rewrite the previous query to be much shorter, and better match the shape of the data. If you put a value in function name position, it implicitly converts it to an elem. You can see that elem is the most used function, and in fact it's what you'll usually be using when munging data, so there's a shortcut. This may sound complicated, but I find it becomes intuitive quite quickly. ![]() In theory we're really just creating and composing a big function - pipeline, so to say - which gets applied to our JSON blob. The second one on the other hand, operates in the context of a single element. The first argument - which should evaluate to the positions we need - gets evaluated in the context of the entire array, that's why keys returns all the indices. ![]() Some functions, like elem, will cut down the context for expressions it evaluates. All functions operate in the context of the JSON we're operating on. Now we have to understand a very important mechanism underlying jql. ![]() cat test.json | jql '(elem "countries" (elem 0 (keys)))' ![]()
0 Comments
Leave a Reply. |