1
2
3
4
5
6
7
8

👏欢迎参加Datawhale夏令营，可以看一下群公告哦，【零基础】跑通一站式baseline！ 我们的学习已经开始了
Step1：30分钟，零基础跑通Baseline获得结果
【baseline链接】：https://www.datawhale.cn/activity/324/learn/191/4314/6/5

Step2：完成打卡，打卡就在教程右侧哦

🎁Tips：打卡完在群公告处有抽奖噢，先到先得～

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

各位同学大家好！欢迎大家参加2025 Datawhale AI 夏令营- 大模型技术方向
请大家先改下群备注，姓名-学校/在职-专业/岗位
-----------------------
【开营预热】强烈推荐【零基础跑通baseline】并参与抽奖！
https://www.datawhale.cn/activity/324/learn/191/4314/6/5

大家跑通baseline后记得在赛事网页提交获得分数，教程右侧进行打卡抽奖哦

【抽奖链接】
https://exn8g66dnwu.feishu.cn/share/base/form/shrcnwWgdu6EheUUd2sY0seC7hf
-------------------------
【开营仪式】人满200人之后我们在本群进行开营仪式哦
-------------------------
【本次学习完全免费，有任何人加你们好友说要付费/推荐付费课程，都是诈骗。如有遇到，欢迎举报信息给官方企微号：小鲸鱼@Datawhale

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

import pandas as pd
video_data = pd.read_csv("origin_videos_data.csv")
comments_data = pd.read_csv("origin_comments_data.csv")
video_data.sample(10) 
//功能：从序列a中随机抽取n个元素，并将n个元素生以list形式返回

comments_data.head() //Return the first n rows.
video_data["text"] = video_data["video_desc"].fillna("") + " " + video_data["video_tags"].fillna("")


import jieba
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import SGDClassifier
from sklearn.svm import LinearSVC
from sklearn.cluster import KMeans
from sklearn.pipeline import make_pipeline


kmeans_predictor = make_pipeline(
    TfidfVectorizer(tokenizer=jieba.lcut), KMeans(n_clusters=2)
)

kmeans_predictor.fit(comments_data[comments_data["user_question"].isin([1])]["comment_text"])


kmeans_cluster_label = kmeans_predictor.predict(comments_data[comments_data["user_question"].isin([1])]["comment_text"])

kmeans_top_word = []
tfidf_vectorizer = kmeans_predictor.named_steps['tfidfvectorizer']
kmeans_model = kmeans_predictor.named_steps['kmeans']
feature_names = tfidf_vectorizer.get_feature_names_out()
cluster_centers = kmeans_model.cluster_centers_

for i in range(kmeans_model.n_clusters):
    top_feature_indices = cluster_centers[i].argsort()[::-1]
    top_word = ' '.join([feature_names[idx] for idx in top_feature_indices[:top_n_words]])
    kmeans_top_word.append(top_word)

comments_data.loc[comments_data["user_question"].isin([1]), "question_cluster_theme"] = [kmeans_top_word[x] for x in kmeans_cluster_label]