Running a Hugging Face LLM on your laptop
Hugging FaceのLarge Language Model（LLM）を自分のマシンで実行する方法について学ぶためのビデオです。この記事では、Hugging FaceのLLMモデルを自分のラップトップで実行する方法について詳しく説明します。
この記事では、こちらから詳細をご覧いただけます。また、実際にコードを実行してみたい方は、こちらのノートブックを参照してください。
他のビデオもありますので、以下からご覧いただけます：
– https://www.youtube.com/watch?v=rfr4p0srlqs
– https://www.youtube.com/watch?v=S2thmwdrYrI
– https://www.youtube.com/watch?v=NFgEgqua-fg
– https://www.youtube.com/watch?v=7BH4C6-HP14
以上が、Hugging FaceのLLMを自分のマシンで実行する方法について魅力的な記事です。AI技術の魅力や応用範囲についても触れるとより読者が興味を持ってくれるかもしれません。
I have done as you say, but running the model pipeline is taking forever to work. It still has not worked, please what can I do?
hi, thanks for the video. May I ask what's the meaning of legacy=False when using the pretrained model?
Thank you for the contents.
I keep following along until about 12 seconds in, where you start typing into something and you say let’s open up age something, and carry on typing into whatever it is you’re typing into – I can’t get that far, I don’t know what to type into
What is the difference w.r.t to using the classical:
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
Thanks in advance!
Hello there Mark i was wondering if i could use this method to download other ai models for example text to image models?
thank you very much for this great explaination
Hi Mark,
Thank you for sharing,
Do you know how to force your example to download files in other path than ".cache/huggingface/hub" ???
os.environ['TRANSFORMERS_CACHE'] = "/media/Tomasz/4T/TRANSFORMERS_CACHE"
%env TRANSFORMERS_CACHE=/media/Tomasz/4T/TRANSFORMERS_CACHE/
both above are ending with models–lmsys–fastchat-t5-3b-v1.0 created in default .cache/huggingface/hub
the last one recipy i found
downloaded_model_path = hf_hub_download(
repo_id=model_id,
filename=filename,
token=HUGGING_FACE_API_KEY,
local_dir = "/media/Tomasz/4T/TRANSFORMERS_CACHE/"
)
also doesnt work.
even have no idea what happend because I cant find enywhere models–lmsys–fastchat-t5-3b-v1.0
Thank you,
cm
I deeply appreciate your video! Although I have a question, does this still works when the model file is a .safetensors or .pth file, not a .bin file? Thank you!
hi Mark – super helpful. can i run all of this in terminal?
Hey, can you tell me about your system info, i am using mac m3 and its not giving any response and running continuously?
i personally found disabling your wifi from a jupyter notebook to be bad ass
Thanks Mark for this video. A quick question- Is this safe to pass some PII data to one of the open source hugging face models that require the hugging face API token ? If No, how can this be resolved in deployment so that there is no risk of data leakage ? Please guide through this.
thx
under the "…" in huggingface there is a "clone this repo" which copies all stuff onto your PC. seems simpler to me.
What is the editor you are using on localhost ?
Absolutely wonderful video! to the point and well explianed! way to go! thanks a lot!
hey um, i don't know if you'll read this in time, but I have a problem:
pytorch_model.bin: 0%| | 0.00/13.5G [00:00<?, ?B/s]
It's bin stuck like that for a long time now
That's Great! Thank you!
Pinned
So many steps missing in this video…
Awesome! Thanks for this video.
Super video!
sir if wifi is on then they model is working properly or not?
I'm probably missing something, but where are you using the downloaded files? you are entering model_id in .from_pretrained(), how is it finding/using the downloaded model?
Seems unnecessarily complex… isn't there like an online space to use this stuff without having to write a bunch of stuff just to download it?
Thank you very much, you helped me a lot
many thanks! however stuck on import check_connectivity from utils…..lol…
thanks sir , however I want to know
1- how one can integrate specific set of models (pre-trained) ones in to Rstudio ? so that one can simply run examples on data "proprietary in my case " locally within R
2- is there a way to ask the inference API for tasks different from the typical sentiment classification of text for example "multi-entity tagging" , "modalities" ….etc
your input is highly appreciated
I am getting an error/ info log from transformers (twice) stating "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained." The model then generates only a bunch of whitespace, no matter the input. I have followed through your steps and made sure the files were downloaded at the expected location. The behavior occurrs both with and without setting legacy=False.
hi. please help me. how to create custom model from many pdfs in Persian language? tank you.
Thanks for this video!!
thanks Mark, very nice video, super clearly put!
could you please suggest, what could be the reason if (when trying to set the wifi off) the output of those lines of code is "ModuleNotFoundError: No module named utils"?