Previously, I introduced a problem I was trying to solve where a large datastructure was being pinned into memory for occasional lookups. This post delves into the implemented solution which pushes it onto disk but retains (relatively) fast lookups. I think using a database or a B-Tree is a good solution to this kind of problem in general, but it was fun and inexpensive to implement this utility, and it turned out to be generally useful. Bear with me if you already understand HashMaps pretty well, because I'm basically describing a HashMap here, but it's a disk-based HashMap.Logically, the data consists of a series of key-value pairs. The keys and values are of variable size, because they contain strings. If we were to write only the values to disk in a binary format, we might have something like this for the JSON example in the previous post: There are two records, at offsets 0x00000000 and 0x000000A4. If we had some way to map a key to one of these offsets, we could seek() to…
Some data structure was pinning over 70MB of heap space in Android Studio. Our developers have limited memory on laptops, and are often upset about memory consumption in general. This behemoth (retained by our internal plugins) was the second largest allocated and pinned single object in the whole of AS's heap.buck project generates IDE project definitions from buck targets. It can be configured it to emit a target-info.json file, which contains simple mappings that look something like this:
{
"//src/com/foo/bar/baz:baz" : {
"buck.type": "java_library",
"intellij.file_path" : ".idea/modules/src_com_foo_bar_baz_baz.iml",
"intellij.name" : "src_com_foo_bar_baz_baz",
"intellij.type" : "module"
"module.lang" : "KOTLIN",
"generated_sources" : [
"buck-out/fe3a3a3/src/com/foo/bar/baz/__gen__",
"buck-out/fe3a3a3/src/…