More hardware but AI

Post #444,265 by crazy 6/24/24 6:16:35 PM 6/24/24 6:16:35 PM Reply	More hardware but AI If I had a thousand bucks throw at something right now to play with for a couple of years and my interest lies in programming. I want to use the AI driven programming co-pilots and play with a bunch. To start off with I accept I'll spend 20 to 40 bucks a month to simply attach to the various AII back ends but in a short while I want to be able to download a couple of models and run them myself. Larger memory and larger context model means more memory. I want as much memory as humanly possible up to 96 GB. I want as many cores as I can afford. I want a true Nvidia co-processor. So does anyone want to toss a configuration out that I might want to start with, possibly from Newegg?
Post #444,266 by pwhysall 6/25/24 2:54:47 AM 6/25/24 2:54:47 AM Reply	www.pcpartpicker.com
Post #444,272 by crazy 6/26/24 1:59:42 PM 6/26/24 1:59:42 PM Reply	Thanks but no I will never build my own PC again. And right now it looks like the difference between combined CPU GPU NPU with 32 GB of RAM on a single chip is the way to go. I really don't care about the GPU with the exception of how it can be used as an AI accelerator. So if there's an npu sitting there to be used then that's probably the way to go. Right now the npus I'm looking at are rated at 16 TOPs. But the ones coming out in 2 months will have 45 .. 50 range. So, if I can wait until September... It looks like the Strix Halo might be shipping in quantity at that point. Which in turn says I should wait for that. Especially since it looks like they are embedding larger quantities of RAM on the chip. It looks like there's 128 GB model. I want I want I want.
Post #444,274 by pwhysall 6/26/24 3:09:19 PM 6/26/24 3:09:19 PM Reply	Eh, just drop £20/mo on a ChatGPT sub You can have that in like seconds. There’s even a desktop app for the Mac.
Post #444,275 by Another Scott 6/26/24 10:11:39 PM 6/26/24 10:11:39 PM Reply	+1 This stuff is changing rapidly. The market seems to be very frothy and who knows what will happen in 6 months with nVidia and all the rest. If I were starting out, I would play around with what's available online and figure out what needs to be done locally and then start picking out parts. https://www.reddit.com/r/LocalLLaMA/comments/1ajijk8/hardware_guide/ https://www.theregister.com/2024/03/17/ai_pc_local_llm/ Etc. Cheers, Scott.
Post #444,279 by crazy 6/28/24 8:59:53 AM 6/28/24 8:59:53 AM Reply	Yep and nope There are two halves of the AI interaction I want. There's the back end on the web immense resources applied to your queries that is always changing and there's no way you can run it at home. I think anthropic Claude is the best at the moment but I'm sure they will leap frog each other so I will probably have multiple accounts depending on who's doing what. I'd probably be trying different co-pilots from the different services as well as the many downloadable ones. But is that account the best use of 99.9% of what I'd want an AI to do at home? No. There's the other part that I want 100% running at home. This part will be fed all my documentation and all my pictures and all my email etc etc etc. This one will be fed all my stories. This part will be monitoring multiple cameras and listening to multiple microphones. Someday I would like to instruct the AI the following: As Me: answer this certain question that is guaranteed to trigger a discussion and possibly an argument with M And then on a different instance of this same ai that has been listening to me and M talk for years: As M: take the other side of the argument This will serve multiple purposes. This will test out if it really understands our personalities and point of view. At that point I can put it in a time capsule and when the computers are powerful enough we can come back in an AI form and live forever together. Of course that won't happen but that seems to be the start of a interesting sci-fi/ fantasy novel. This part would be a privacy nightmare if it ended up on somebody's desk at Google or openai or anthropic. So this stuff gets run at home behind the firewall. And the firewall better be watching for this AI doing stuff I don't want it doing. You may laugh at that, but the Chinese have released a coding AI that is rated very high and I might be stupid enough to try it. Free and downloadable and almost as good as Claude. Which means I better have a really good firewall watching it. Good thing those devices are cheap. We have various PC builders on Craigslist that can put together something for me. The following looks like a good start: https://seattle.craigslist.org/tac/sys/d/tacoma-geforce-rtx-3070-gaming-pc/7760094617.html I sent him an email to see how much memory he can put in it and does he deliver to me? The 3070 looks like the best card out right now as far as price performance is concerned. Lots of cuda and other cores to work with. I'm trying to keep it under $1,000 and this is fitting the bill.
Post #444,280 by CRConrad 6/28/24 10:11:09 AM 6/28/24 10:11:09 AM Reply	So basically you want AI to outsource... Arguing with your wife? Isn't that A) cheating?, and B) futile, when she finds out and gets a bigger one? :-D -- Christian R. Conrad The Man Who Apparently Still Knows Fucking Everything Mail: Same username as at the top left of this post, at iki.fi
Post #444,281 by drook 6/28/24 12:26:41 PM 6/28/24 12:26:41 PM Reply	That won't be you It might feel like you to people who interact with it, but the you that's talking now would be well and truly dead. How does it benefit this you for there to be a different you once you're gone? -- Drew
Post #444,282 by crazy 6/28/24 6:30:02 PM 6/28/24 6:30:02 PM Reply	Just to be able to say this me versus a different me sounds fun! Of course it won't be me. But I told the boy I wouldn't write a book. I didn't say I wouldn't leave behind an AI of myself. Imagine my AI calling him up and lecturing him on his life choices.
Post #444,285 by drook 6/28/24 11:00:47 PM 6/28/24 11:00:47 PM Reply	Uhh ... ... we can come back in an AI form and live forever together. Kind of sounds like that's how you're thinking. -- Drew
Post #444,286 by crazy 6/29/24 12:50:29 AM 6/29/24 12:50:29 AM Reply	And I immediately said that's a fantasy
Post #444,269 by drook 6/25/24 1:35:17 PM 6/25/24 1:35:17 PM Reply	Ask one of the LLMs to specify a configuration to run LLMs -- Drew
Post #444,273 by crazy 6/26/24 2:03:39 PM 6/26/24 2:03:39 PM Reply	And have it lie to me? It's my responsibility to determine use case and what is a reasonable response time. I would really like to see it as a coding co-pilot and see how it analyzes and offers suggestions to rewrite my code. I'm looking at a very focused use case for that. But I also wanted to do image analysis and security and light control and.... You get the point. One of the issues is I'll probably have to end up with two separate systems. The back end AI engine and the front end desktop that's got the real video card for visual purposes. And that's got a lot of memory so we can run the programming environments which even before AI sucked down a shitload of memory. It's up to me to determine usability. I might end up with a toy that is not useful at all. That's okay, I would just like a reasonable starting point.
Post #444,277 by CRConrad 6/27/24 2:37:04 AM 6/27/24 2:37:04 AM Reply	Doesn't that mean just drop the whole idea? If you won't ask an LLM because LLMs lie, then what's the use of building a system to run LLMs in the first place? -- Christian R. Conrad The Man Who Apparently Still Knows Fucking Everything Mail: Same username as at the top left of this post, at iki.fi

Welcome to IWETHEY!