• 3 Posts
  • 31 Comments
Joined 1 year ago
cake
Cake day: June 19th, 2023

help-circle

  • It cannot “analyze” it. It’s fundamentally not how LLM’s work. The LLM has a finite set of “tokens”: words and word-pieces like “dog”, “house”, but also like “berry” and “straw” or “rasp”. When it reads the input it splits the words into the recognized tokens. It’s like a lookup table. The input becomes “token15, token20043, token1923, token984, token1234, …” and so on. The LLM “thinks” of these tokens as coordinates in a very high dimensional space. But it cannot go back and examine the actual contents (letters) in each token. It has to get the information about the number or “r” from somewhere else. So it has likely ingested some texts where the number of "r"s in strawberry is discussed. But it can never actually “test” it.

    A completely new architecture or paradigm is needed to make these LLM’s capable of reading letter by letter and keep some kind of count-memory.


  • I have no idea how to fix the problem, but I’ve read somewhere that burn (a relatively new machine learning framework in Rust) is capable of loading models like stable diffusion. As Burn is built with webGPU and all the shader transpiler-stuff that comes with it doesn’t that mean that it can also run easily on (even older) AMD cards? I think what’s lacking is equal performance as nvidia drivers are heavily optimized already.

    Maybe someone knows more here?






  • In this case it’s a giant housing shortage though. The city (and large surrounding area) is Freiburg in the south. Rents are so expensive and available flats are so rare that companies don’t find workers who could actually live there. Also: the comparably good loans don’t mean much when it’s only channeled into a greedy landlord’s pockets.

    Edit: oh no i was wrong it’s Nuremberg - their public transport organization is also “VAG”. But Freiburg has a huge labor shortage due to unaffordable housing and housing shortage.


  • Have you been to Europe? Have you walked the streets of Paris? The US was built with enough space being everywhere. American roads are wider, cities are mostly built like square-grids of roads built in a time when cats existed whereas European cities emerged in the middle ages. They’re tightly packed with little extra space. Sometimes (very rarely) here there are old Cadillacs that can be rented for weddings. Seeing one of these cars on the street is an unreal experience. They’re just so huge. They don’t fit on the streets here - and those are cars from the 60s or 70s. Everything seems tiny compared to them. From a European perspective it’s really stupid to build such large vehicles as driving and parking it is much more complicated when everything is build for small cars. Now that SUVs are becoming popular here too it’s just a really annoying. Less parking space per vehicle etc. On cities like Paris - one of the tightest city on Europe this is just annoying. And i haven’t even written about fuel consumption. Paris has had huge problems with smog in recent years.





  • Oh, i have a brilliant one:

    A few years ago i spent a lot of time converting .flac-files into .ogg-files in order to put on my oldschool iPod. As I did a lot of repetitive typing - entering $dir / for file in flac ; do convert etc / mkdir -p $somewhere/$artist/$album / mv $somewhere/.ogg->$new_dir/ and so on - I thought: “hm lets just write a loop over loops for all the artists here and then all the albums and at the same time create the nested directories somewhere else… hm actually in the home directory… and later love everything on the iPod at once.”

    so i was in my music folder with the artists-folders i wanted to convert. i did something wrong

    So i did my complicated script directly in the shell. I made something wrong and instead of creating a folder “~/artist/album” I created 3 folders in my current working directory: “~”, “artist” and “album”. hmph dammit gotta try again… but first : i have to clean up these useless folders in the current dir. so i type of course this: "$ rm -r ~ artist album " after about 5 seconds of wondering why it took so long i realized my error. o_O I stopped the running command, but it was (of course) too late and i bricked my current installation. All the half-deleted config files made or impossible to start normally and extremely tedious to repair it by hand, so i reinstalled.