a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment
rob05c  ·  3250 days ago  ·  link  ·    ·  parent  ·  post: Devski Update: Notifications in SQL, Post Performance Fix

Data was originally stored as serialised s-expressions. So, files containing e.g. `(publication (id 1234) (user foo) (tag bar) (tag2 baz) …)`. This worked because all data was loaded into memory on startup. Keeping everything in memory is both not scalable, and expensive for hosting.

If the data isn't loaded on start and kept in memory, It's untenably slow to load and search files like that for specific data you need, when you need it. SQL is fast to query. SQL is also more flexible: we can change SQL servers, or even pay for separate SQL hosting if it's cheaper.

Some of our data fits a 'NoSQL' document store model, but not all, so we'd have to keep two databases, or deal with non-relational pain for our relational data. Further, if we think we need the performance of non-relational storage, PostgreSQL is still faster than Mongo.

Hubski is moving to Racket for flexibility. Arc is a terrible language. It's poorly designed, and even more poorly documented, and completely lacks many features such as SQL. Racket is very well documented, and has libraries for everything we need. While mk has been writing Arc for years and has a pretty good handle on it, for everyone else, it's a nightmare to figure out what obscure, undocumented macros and functions do. Having known neither before I started on Hubski, it easily takes me 5–10× as long to write Arc as Racket. Arc is also poorly mantained; if there were a bug in the interpreter, we'd likely be stuck with it. Racket is actively mantained by a large community. In short, Arc is dead. We're moving to a live language.