ai-learn
· 2 min read
-
https://github.com/search?q=word2id+batch_size+seq_len+zip_longest&ref=opensearch&type=code
-
https://community.openai.com/t/foundational-must-read-gpt-llm-papers/197003
-
https://community.openai.com/t/foundational-must-read-gpt-llm-papers/197003
refs site
- https://github.com/mikaizhu/Notes/blob/f04d80456dde566abbd7e8cd1e3528cef1bc6f74/Deep_learning/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/TencentAds.md
- https://www.sharetechnote.com/html/NN/Handbook_MachineLearning_Index.html
- https://pytorch.org/tutorials/
- https://pytorch.org/tutorials/beginner/basics/intro.html
- https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-embeddings-tutorial-py
- ai for begginer https://github.com/microsoft/AI-For-Beginners
- https://www.baeldung.com/cs/category/ai
To remember
- one-hot encoding
- torch.nn.Liner
- torch.nn.Embedding
torch.tensor([[1,2,4,5]])
weights = torch.rand(10, 3)
net = torch.nn.liner(10,2)
print(net,net.h)
torch.mm
AI > machine learning > deep learning
pytorch.nn.Embedding
- https://www.jianshu.com/p/63e7acc5e890
- https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-embeddings-tutorial-py
- https://discuss.pytorch.org/t/how-does-nn-embedding-work/88518/3
理解Epoch, Batch, and Mini-batch
- Gradient Descent
- Batch Gradient Descent
- Stochastic Gradient Descent
- Mini-Batch Gradient Descent
- 比如有个2000个sample
- 将2000个sample整体 一次性输入 (input net train&backward feedback) 就是Batch Gradient Descent
- 2000个数据one by one共2000次分别(input net train&backward feedback) 就是 stochastic gradient descent
- 2000个数据份 n组 每组2000/n个sample(input net train&backward feedback) 就是Mini-Batch Gradient Descent
- 上面三个方式，一次整体sample train完称为一个epoch
- 要经过多次epoch才能达到(Gradient Descent)剃度下降效果
similarity相似度 | diversity index多样性指数
- Cosine similarity #向量i和j之间的向量夹角大小 越小越相似
- Pearson Correlation Coefficient(皮尔逊相关系数) #与余弦相似度相比,加入了用户平均分对各独立评分进行修正,减小了用户评分偏置的影响
- Euclidean Distance #m维空间中两点之间的距离
- Jaccard Index(Jaccard similarity coefficient) #两个集合A和B的交集在其二者的并集中所占的比例
- Jaccard Distance #两个集合中不同元素占所有元素的比例来衡量两个集合的区分度
