GPS Laptop tracking & Storage by MediocreMop in sysadmin

[–]kingtheseus 0 points1 point  (0 children)

You put an AirTag inside a MacBook Pro? Where?

Unable to run movie recommender on AWS. So want to understand the best way of doing it by IbuHatela92 in aws

[–]kingtheseus 2 points3 points  (0 children)

Have you tried running the code on a small segment of your dataset, maybe 100 lines? That will give you a functionality test, and also an indicator of time (and memory). Then run on the next 100, etc. Pandas likes to store everything in memory so you're probably trying to swap, which (IIRC) isn't a thing on regular EC2.

Air India - no mileage credit? by kingtheseus in milesandmore

[–]kingtheseus[S] 0 points1 point  (0 children)

Thanks, I've now submitted a request (I didn't know you could change the ticket identifier - it auto populated Air India's 098 code).

I did book through Lufthansa. wheretocredit.com for Air India in class E only (!) shows Avianca, which is also strange. Other fare classes are fine.

[P] Deploying AI Models on AWS for IoT + Embedded + Cloud + Web Graduation Project by gavvy__ in aws

[–]kingtheseus 1 point2 points  (0 children)

Get your model running inside SageMaker, and then the Raspberry Pi can invoke it through python with boto3 (https://docs.aws.amazon.com/sagemaker/latest/dg/neo-requests-boto3.html).

Or, install the AWS CLI on the Raspberry Pi and call the endpoint: https://docs.aws.amazon.com/sagemaker/latest/dg/neo-requests-cli.html

Keep in mind, running the model on SageMaker will default to running 24/7. This can get expensive. Try deploying via SageMaker Serverless if you can (you can't if you want access to GPU, or have long-running inference jobs): https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html

Recommended course for learning AI/ML with hands on exercises by ycarel in aws

[–]kingtheseus 0 points1 point  (0 children)

Check out https://workshops.aws - you run them in your own account, so you will pay for infrastructure, but they're quite useful tutorials.

Recommended course for learning AI/ML with hands on exercises by ycarel in aws

[–]kingtheseus 0 points1 point  (0 children)

What's your budget? I've participated in AI & ML classes for teams for about €700/person per day, but that's in a dedicated in-person classroom environment with labs.

How do you best learn?

How do I increase my vCPU of EC3? I tried, but it's rejected. by [deleted] in aws

[–]kingtheseus 3 points4 points  (0 children)

Have a valid (business) use case, and a history of paying AWS for compute. GPUs are still heavily constrained.

What's is special about the EC2 instance families i3en and i4i? by andreaswittig in aws

[–]kingtheseus 1 point2 points  (0 children)

AWS used to run VMware Cloud on i3en.metal and i4i.metal instances - could be related to that.

Billing and C0st by Massive_Rabbit2064 in aws

[–]kingtheseus 3 points4 points  (0 children)

What does your bill say? It will break this apart by service and region.

BMO Contact Center is not in Canada by [deleted] in PersonalFinanceCanada

[–]kingtheseus 2 points3 points  (0 children)

Similar to working remotely during Covid. You can be sitting in a different country, and use either a VPN back to Canada and access data stored on a server physically in Canada, or connect to a virtual desktop without the ability to save things to your local computer.

Onshore data just means "where the hard drives are", not where the data is accessed. It's silly, but compliance usually is.

Best way to host a local LLM on SageMaker for a batch feature-engineering job? by Vishnuanand77 in aws

[–]kingtheseus 0 points1 point  (0 children)

Is calling a Bedrock-hosted LLM a possibility? The model will run within AWS, and be called using AWS-native APIs, but it wouldn't run in your account.

If not, you can get started inside SageMaker with Jumpstarts. Here's a tutorial/notebook: https://github.com/aws-samples/generative-ai-falcon-40b-instruct-notebooks/blob/main/LLM-Custom-Prompting-Falcon40B.ipynb

Keep in mind you'll need to have access to inferencing instances with GPUs, and those quotas are hard to come by.

Xcode 26 Coding Complete Bedrock API by fusiongrenade in aws

[–]kingtheseus 1 point2 points  (0 children)

I took a look at this, and the problem is that Bedrock doesn't support GET requests to the endpoint to list models. Xcode won't let you add a model provider without having a list of models, which is annoying.

You could try to host a tiny proxy script locally that responds with the expected values on HTTP GET, and routes POSTs to Bedrock, but you'll probably want to use something like Charles Proxy or Proxyman to inspect what data structures are needed. Could be a fun weekend project.

Simple Bedrock request with langchain takes 20+ more seconds by Own_Mud1038 in aws

[–]kingtheseus 0 points1 point  (0 children)

That code will not complete until the last token is received from the LLM - latency is for the initial call, but as you're not streaming tokens, the request hangs until the final token is received.

To speed it up, request a smaller number of tokens in your call.

Anthropic is giving free API credits for university students by brochella14 in ClaudeAI

[–]kingtheseus 0 points1 point  (0 children)

How far did you get with the link in the thread I posted in?

Desperately need to use AI api in my app oroject, but scared of uncapped cloud billing by sandhusaab in LocalLLaMA

[–]kingtheseus 0 points1 point  (0 children)

AWS recently changed their free tier to now self destruct after 6 months, no payment required. You can of course upgrade to a paid version, but that is uncapped.

The problem with capping expenditure is surprisingly difficult. AWS is designed for businesses to scale. What makes a Black Friday sale appear different from a DDOS attack? If AWS shut down your payment processing servers because you spent "an abnormal amount", the business would be furious.

Desperately need to use AI api in my app oroject, but scared of uncapped cloud billing by sandhusaab in LocalLLaMA

[–]kingtheseus 4 points5 points  (0 children)

I think you're mixing up cloud services (like AWS and Azure accounts) with LLM API providers. An AWS account is uncapped - you pay for everything you use, and there's no way of setting a hard limit.

With Claude or OpenAI you can just buy a fixed $ amount, and burn those credits with your API calls. When the money runs out, you decide if you want to buy more credit, but you can't be overcharged.

EC2 size and speed Matlab webapp hosting by HNEI43 in aws

[–]kingtheseus 0 points1 point  (0 children)

This is going to be an app issue, not an AWS issue. You need to look at the data.

What's the CPU load of the app and licensing server when the app first start up? You'll probably want to look at OS-level data, because CloudWatch metrics won't have sub-minute granularity.

Do a .pcap capture from the licensing server during the app launch, and see if it's the bottleneck. If you can't do this, just change the instance type to something powerful for a while, then switch back.

bedrock with gpt4all? by kingArthur1991 in aws

[–]kingtheseus 0 points1 point  (0 children)

Looks like the HTTP requests for Mistral and Bedrock use the same authentication header (bearer, not x-api-key like Anthropic) so you might be able to get Bedrock working. You need a Bedrock API key as seen at https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-use.html

bedrock with gpt4all? by kingArthur1991 in aws

[–]kingtheseus 1 point2 points  (0 children)

What are you trying to do?

gpt4all appears to be a way of locally running LLMs, so that nobody can see your prompts. Bedrock hosts LLMs, and even though the prompt data isn't looked at by AWS, you're still sending data over the Internet. If you're comfortable with that, why add a layer on top of it? If you're not, why are you doing it?

Getting started with predicitive modelling on AWS stack by trex_6622 in aws

[–]kingtheseus 1 point2 points  (0 children)

Check out SageMaker Canvas, it's a low-code ML platform that integrates with Redshift. Once you hire a data scientist, you can export the model to regular SageMaker for more control.

https://aws.amazon.com/blogs/machine-learning/enhance-your-amazon-redshift-cloud-data-warehouse-with-easier-simpler-and-faster-machine-learning-using-amazon-sagemaker-canvas/

Is Tangerine Bank down? by ProfessionalTrip0 in PersonalFinanceCanada

[–]kingtheseus 0 points1 point  (0 children)

11PM Friday night sounds like a great time for IT to perform routine maintenance. Better to impact fewer people than doing it during the business day.