You should definitely benchmark it using a simulation of what your data might look like. It should not be that hard. Just make script, that creates bunch of files similar to your data.
Right, it’s just kind of a thing to think about. If your program is something that might conceivably be used of sshfs (as an example) … this is probably not a great option for your program’s configuration.
The data structure and parser is not formed the same way as the json, where you have to parse the whole thing. So you don’t have to. You just open the files you need read their content. It is a bit more difficult at first since you can’t just translate a whole struct directly, but it pays for itself when you want to migrate the data to a new format. So if your structure never changes, probably those formats are easier.
Well a very common thing is to create a “config” object that lives in the long running process (and in some cases can be reloaded without restarting the program).
That model also saves you from unnecessary repeated IO operations (without one off caching and reloading mechanisms) and allows you to centralize any validation (which also means you can give configuration errors on start up).
I do wish various formats were more “streaming” friendly, but configuration isn’t really one of them.
In a lot of languages moving between formats is also fairly trivial because the XYZ markup parser parses things into an object map and the ZYK markup writer can write an object map into ZYK format.
Maybe I’m not understanding what you mean by migrating the data to a new format though.
OK so, for example if you have to change the structure of the configuration file, in a statically typed language. You have to have two representation of the data, the old one, and the new one.You have to first deserialize the data, in the old format, then convert it back to the new format, then replace the old files.
The FAMF alternative, allows you just to easily use copy and paste and delete to achieve the same goal.
Please keep in mind that you can just make configuration data structure that you can keep in-memory. It is just that the representation of the persisted information is spread between different files and not just one file.
Right, it’s just kind of a thing to think about. If your program is something that might conceivably be used of sshfs (as an example) … this is probably not a great option for your program’s configuration.
Well a very common thing is to create a “config” object that lives in the long running process (and in some cases can be reloaded without restarting the program).
That model also saves you from unnecessary repeated IO operations (without one off caching and reloading mechanisms) and allows you to centralize any validation (which also means you can give configuration errors on start up).
I do wish various formats were more “streaming” friendly, but configuration isn’t really one of them.
In a lot of languages moving between formats is also fairly trivial because the XYZ markup parser parses things into an object map and the ZYK markup writer can write an object map into ZYK format.
Maybe I’m not understanding what you mean by migrating the data to a new format though.
OK so, for example if you have to change the structure of the configuration file, in a statically typed language. You have to have two representation of the data, the old one, and the new one.You have to first deserialize the data, in the old format, then convert it back to the new format, then replace the old files. The FAMF alternative, allows you just to easily use copy and paste and delete to achieve the same goal. Please keep in mind that you can just make configuration data structure that you can keep in-memory. It is just that the representation of the persisted information is spread between different files and not just one file.