“The leverage” to do what exactly? Put in someone who will be way worse? How does that help the left accrue power or accomplish our goals? If you think the Democratic Party’s takeaway from the left tanking a major election will be “we need to move left more” I have a bridge to sell you. We are not a majority, which means we need to form coalitions. We can’t do that with a reputation of blowing up everyone’s shit when we don’t get our way. We do it by showing how successful the party is when they listen to us and include us. No, this time we don’t have a particularly left candidate to vote for. Yes, it all sucks. But I have yet to see a concrete explanation of how picking or allowing “far right fascist” over “moderate” has any benefit in the short or long term. To my eyes, it just causes vulnerable people here and around the world to suffer.

Archive Team looked at this about 10 years ago and found it basically impossible. It was around 14 petabytes of information to fetch, organize, and distribute at the time.

https://wiki.archiveteam.org/index.php/INTERNETARCHIVE.BAK

Do you think electing Trump will be read as “wow, the US is taking a principled stance on Palestinian rights” by the world?

“Lossless” has a specific meaning, that you haven’t lost any data, perceptible or not. The original can be recreated down to the exact 1s and 0s. “Lossy” compression generally means “data is lost but it’s worth it and still does the job” which is what it sounds like you’re looking for.

With images, sometimes if technology has advanced, you can find ways to apply even more compression without any more data loss, but that’s less common in video. People can choose to keep raw photos with all the information that the sensor got when the photo was taken, but a “raw” uncompressed video would be preposterously huge, so video codecs have to throw out a lot more data than photo formats do. It’s fine because videos keep moving, you don’t stare at a single frame for more than a fraction of a second anyway. But that doesn’t leave much room for improvement without throwing out even more, and going from one lossy algorithm to another has the downside of the new algorithm not knowing what’s “good” visual data from the original and what’s just compression noise from the first lossy algorithm, so it will attempt to preserve junk while also adding its own. You can always give it a try and see what happens, of course, but there are limits before it starts looking glitchy and bad.

I know TiddlyWiki quite well but have only poked at Logseq, so maybe it’s more similar to this than I think, but TiddlyWiki is almost entirely implemented in itself. There’s a very small core that’s JavaScript but most of it is implemented as wiki objects (they call them “tiddlers,” yes, really) and almost everything you interact with can be tweaked, overridden, or imitated. There’s almost nothing that “the system” can do but you can’t. It’s idiosyncratic, kind of its own little universe to be learned and concepts to be understood, but if you do it’s insanely flexible.

Dig deep enough, and you’ll discover that it’s not a weird little wiki — it’s a tiny, self-contained object database and web frontend framework that they have used to make a weird little wiki, but you can use it for pretty much anything else you want, either on top of the wiki or tearing it down to build your own thing. I’ve used it to make a prediction tracker for a podcast I follow, I’ve made my own todo list app in it, and I made a Super Bowl prop bet game for friends to play that used to be spreadsheet-based. For me, it’s the perfect “I just want to knock something together as a simple web app” tool.

And it has the fun party trick (this used to be the whole point of it but I’d argue it has moved beyond this now) that your entire wiki can be exported to a single HTML file that contains the entire fully functional app, even allowing people to make their own edits and save a new copy of the HTML file with new contents. If running a small web server isn’t an issue, that’s the easiest way to do it because saving is automatic and everything is centralized, otherwise you need to jump through some hoops to get your web browser to allow writing to the HTML file on disk or just save new copies every time.

If you run the Node.js version, that’s all handled for you. It’s only if you want to do the party trick of keeping it all in a single HTML file that you need to worry about a plugin or anything like that. And even then, the server version exports to a standalone HTML file with one or two clicks.

I think that joke’s been around for a while, but there is the Terry Pratchett line about how if you had a button with a sign next to it saying “pressing this button will end the world, do not touch,” the ink wouldn’t even have time to dry.

How many do you think it is, and how much more acceptable is that number than this one?

“There was a particular bad guy near them” and “they all probably have bad opinions about Jews” are not sufficient justifications for indiscriminately bombing innocent people. What if there had been an Israeli leader at that rave? People in both refugee camps and at a music event should be able to exist without fear that they’ll die because they were near the wrong person. One seems to provoke a different reaction than the other for some reason though, and that might be worth thinking about.

These models aren’t great at tasks that require precision and analytical thinking. They’re trained on a fairly simple task, “if I give you some text, guess what the next bit of text is.” Sounds simple, but it’s incredibly powerful. Imagine if you could correctly guess the next bit of text for the sentence “The answer to the ultimate question of life, the universe, and everything is” or “The solution to the problems in the Middle East is”.

Recently, we’ve been seeing shockingly good results from models that do this task. They can synthesize unrelated subjects, and hold coherent conversations that sound very human. However, despite doing some things that up until recently only humans could do, they still aren’t at human-level intelligence. Humans read and write by taking in words, converting them into rich mental concepts, applying thoughts, feelings, and reasoning to them, and then converting the resulting concepts back into words to communicate with others. LLMs arguably might be doing some of this too, but they’re evaluated solely on words and therefore much more of their “thought process” is based on “what words are likely to come next” and not “is this concept being applied correctly” or “is this factual information”. Humans have much, much greater capacity than these models, and we live complex lives that act as an incredibly comprehensive training process. These models are small and trained very narrowly in comparison. Their excellent mimicry gives the illusion of a similarly rich inner life, but it’s mostly imitation.

All that comes down to the fact that these models aren’t great at complex reasoning and precise details. They’re just not trained for it. They got through “life” by picking plausible words and that’s mostly what they’ll continue to do. For writing a novel or poem, that’s good enough, but math and physics are more rigorous than that. They do seem to be able to handle code snippets now, mostly, which is progress, but in general this isn’t something that you can be completely confident in them doing correctly. They make silly mistakes because they aren’t really thinking it through. To them, there isn’t really much difference between answers like “that date is 7 days after Christmas” and “that date is 12 days after Christmas.” Which one it thinks is more correct is based on things it has seen, not necessarily an explicit counting process. You can also see this in things like that case where someone tried to use it to write a legal brief, where it came up with citations that seemed plausible but were in fact completely made up. It wasn’t trained on accurate citations, it was trained on words.

They also have a bad habit of sounding confident no matter what they’re saying, which makes it hard to use them for things you can’t check yourself. Anything they say could be right/accurate/good/not plagiarized, but the model won’t have a good sense of that, and if you don’t know either, you’re opening yourself up to risk of being misled.

There just isn’t much use for an approach like this, unfortunately. TypeScript doesn’t stand alone enough for it. If you want to know how functions work, you need to learn how JavaScript functions work, because TypeScript doesn’t change that. It adds some error checking on top of what’s already there, but that’s it.

An integrated approach would just be a JavaScript book with all the code samples edited slightly to include type annotations, a heavily revised chapter on types (which would be the only place where all those type annotations make any difference at all, in the rest of the book they’d just be there, unremarked upon), and a new chapter on interoperating with vanilla JavaScript. Seeing as the TypeScript documentation is already focused on those exact topics (adding type annotations to existing code, describing how types work, and how to work with other people’s JavaScript libraries that you want to use too), you can get almost exactly the same results by taking a JavaScript book and stapling the TypeScript documentation to the end of it, and it’d have the advantage of keeping the two separate so that you can easily tell what things belong to which side.

Protect me from knowing what I don’t need to know. Protect me from even knowing that there are things to know that I don’t know. Protect me from knowing that I decided not to know about the things that I decided not to know about. Amen.

Lord, lord, lord. Protect me from the consequences of the above prayer. Amen.

That’s part of the point, you aren’t necessarily supposed to have an empty mind the whole time. I mean, if you can do that, great, but you aren’t failing if that’s not the case.

Imagine that your thoughts are buses, and your job is to sit at the bus stop and not get on any of them. Just notice them and let them go by. Like a bus stop, you don’t really control what comes by, but you do control which ones you get on board and follow. If you notice that you’ve gotten on a bus, that’s fine, just get off of it and go back to watching. Interesting things can happen if you just watch and notice which thoughts go by, and it’s good practice for noticing what you’re thinking and where you’re going and taking control of it yourself when it’s somewhere you don’t want to go.

I use TiddlyWiki for, well, a bunch of my projects, but primarily for my task management. You can use it as a single HTML file, which contains the entire wiki, your data, its own code, all of it, and of course use it in any browser you like. Saving changes is a bit of a pain until you find a browser extension or some other way of enabling more seamless editing than re-saving the edited wiki as another single HTML file, but there are many solutions to that as described on their site above.

The way I use it, which is more technical but also logistically simpler, is by running their very minimal Node.JS server which you can just visit and use in any browser which takes care of saving and syncing entirely.

The thing I like about TiddlyWiki is that although on its surface it’s a quirky little wiki with a fun party trick of fitting into an HTML file, what it actually is is a self-contained lightweight object database with a simple yet powerful query language and miniature front-end web development environment which they have used to implement a quirky little wiki. Each “article” is an object that is taggable and has key/value data, and “widgets” can be used in the text to edit and display that data, pulling from the “database” using filters. You can use it to make simple web apps for yourself and they come together very quickly once you know what you’re doing, and the entire thing is a demonstration of a complex web app that is also possible. The wiki’s implemented entirely using those same tools, and everything is open for you to tweak and edit to your liking.

I moved a Super Bowl guessing/fake gambling game that I run from a form and spreadsheet to a TiddlyWiki and now I can share an online dashboard that live updates for everyone and it was decently easy to make and works really well. With my task manager, I recently decided to add a feature where I can set an “agenda” value on any task, and they all show up in one place, so I could set it as “Boss” and then quickly see everything I wanted to bring up in our next 1 on 1 meeting. It took just a few minutes to add the text box to anything that gets tagged “Task” and then make another page that collected them all and displayed them in sections.

This is the key with all the machine learning stuff going on right now. The robot will create something, but none of them have a firm understanding of right, wrong, truth, lies, reality, or fiction. You have to be able to evaluate its output because you have no idea if the robot’s telling the truth or not at that moment. Images are pretty immune to this because everyone can evaluate a picture for correctness or realism, and even if it’s a misleading photorealistic image, well, we’ve already had Photoshops for a long time. With text, you always have to keep in mind that the robot might be low quality or outright wrong, and if you aren’t equipped to evaluate its answers for that, you shouldn’t be using it.

There is never going to be a case where the world misses the answer to the ultimate question of life, the universe, and everything because it was said by a Nazi and everyone refused to listen to the Nazi. When it’s clearly straight up propaganda, it’s perfectly rational to dismiss it due to the source and not investigate further. If there’s a valid and useful point to be made, it’ll get made in more respectable sources too and then it might be time to pay attention. Plus, even if they do cite sources, it’s hard to spot where they’ve twisted or lied about those sources, but it’s really easy for the propagandist to spout whatever nonsense they believe because they don’t care about the truth. That asymmetry is good for the Nazi and bad for decent people, and the way to fix that is don’t waste your time carefully investigating and critiquing Nazi bullshit.

The doom and gloom predictions have always been about slow but inexorable changes in the climate. Not that suddenly a mega hurricane is going to rip Florida out of the ground and toss it into the ocean, but that weather is going to get worse and more extreme, that sea levels will rise, and more and more places will gradually become uninhabitable as conditions get worse. There won’t be single things that you can point to and say “that one was global warming”, it’s about trends that are harmful for us in the long term. If you eat a chocolate bar’s worth more calories than you burn every day, it sounds like doom and gloom to say you’ll gain 200 pounds if you don’t change anything, and you won’t be able to point to any one meal as something to be concerned about because that’s not really out of the ordinary for a day… but slowly and steadily, you’ll gain weight, and if nothing changes you will get there eventually.

And even though you aren’t owed dramatic destruction, and shouldn’t require it to believe the thousands of people who study this as their life’s work and all agree that things are dire and not getting better fast enough… you’ve literally just lived through the hottest twenty or so days in recorded history. Is that a coincidence, do you think?

I hope I don’t come across as too cynical about it :) It’s pretty amazing, and the things these things can do in, what, a few gigabytes of weights and a beefy GPU are many, many times better than I would’ve expected if you had outlined the approach for me 2 years ago. But there’s also a long history of GAI being just around the corner, and we do keep turning corners and making useful progress, but it’s always still a ways off after each leap. I remember some people thinking that chess was the pinnacle of human intelligence, requiring creativity and logic to succeed, and when computers blew past humans at chess, it became clear that no, that’s still impressive but you can get good at chess without really getting good at anything else.

It might be possible for an ML model to assemble itself into general intelligence based solely on being fed words like we’re doing, it does seem like the data going in contains enough to do that, but getting that last 10% is going to be hard, each percentage point much harder than the last, and it’s going to require more rigorous training to stop them from skating by with responses that merely come close when things get technical or precise. I’d expect that we need more breakthroughs in tools or techniques to close that gap.

It’s also important to remember that as humans, we’re inclined to read consciousness and intent into everything, which is why pretty much every pantheon of gods includes one for thunder and lightning. Chatbots sound human enough that they cross the threshold for peoples’ brains to start gliding over inaccuracies or strange thinking or phrasing, and we also unconsciously help our conversation partner by clarifying or rephrasing things if the other side doesn’t seem to be understanding. I suppose this is less true now that they’re giving longer responses and remaining coherent, but especially early on, the human was doing more work than they realized keeping the conversation on the rails, and once you started seeing that it removed a bit of the magic. Chatbots are holding their own better now but I think they still get more benefit of the doubt than we realize we’re giving them.

Thanks for that article, it was a very interesting read! I think we’re mostly agreeing about things :) This stood out to me from there as an encapsulation of the conversation:

I don’t think LLMs will approach consciousness until they have a complex cognitive system that requires an interface to be used from within – which in turn requires top-down feedback loops and a great deal more complexity than anything in GPT4. But I agree with Will’s general point: language prediction is sufficiently challenging that complex solutions are called for, and these involve complex cognitive stratagems that go far beyond anything well described as statistics.

“Statistics” is probably an insufficient term for what these things are doing, but it’s helpful to pull the conversation in that direction when a lay person using one of those things is likely to assume quite the opposite, that this really is a person in a computer with hopes and dreams. But I agree that it takes more than simply consulting a table to find the most likely next word to, to take an earlier example, write a haiku about Danny DeVito. That’s synthesizing two ideas together that (I would guess) the model was trained on individually. That’s very cool and deserving of admiration, and could lead to pretty incredible things. I’d expect that the task of predicting words, on its own, wouldn’t be stringent enough to force a model to develop “true” intelligence, whatever that means, to succeed during training, but I suppose we’ll find out, and probably sooner than we expect.

I would agree that we are also very complicated statistical models, there’s nothing magical going on in the human brain either, just physics which as far as we know is math that we could figure out eventually. It’s a massively huge order of magnitude leap in complexity from current machine learning models to human brains, but that’s not to say that the only way we’ll get true artificial intelligence is by accurately simulating a human brain, I’d guess that we’ll have something that’s unambiguously intelligent by any definition well before we’re capable of that. It’ll be a different approach from the human brain and may think and act in alien or unusual ways, but that can still count.

Where we are now, though, there’s really no reason to expect true intelligence to emerge from what we’re currently doing. It’s a bit like training a mouse to navigate a maze and then wondering whether maybe the mouse is now also capable of helping you navigate your cross-country road trip. “Well, you don’t know how it’s doing it, maybe it has acquired general navigation intelligence!” It can’t be disproven, I guess, but there’s no reason to think that it picked up any of those skills because it wasn’t trained to do any of that, and although it’s maybe a superintelligent mouse packing a ton of brainpower into a tiny little brain, all our experience with mice would indicate that their brains aren’t big enough or capable of that regardless of how much you trained them. Once we’ve bred, uh, mice with brains the size of a football, maybe, but not these tiny little mice.