Parcourir la source

smooth term weight (#3510)

### What problem does this PR solve?

#3499

### Type of change

- [x] Performance Improvement
tags/v0.14.0
Kevin Hu il y a 11 mois
Parent
révision
289034f36e
1 fichiers modifiés avec 2 ajouts et 2 suppressions
  1. 2
    2
      rag/nlp/term_weight.py

+ 2
- 2
rag/nlp/term_weight.py Voir le fichier

@@ -228,7 +228,7 @@ class Dealer:
idf2 = np.array([idf(df(t), 1000000000) for t in tks])
wts = (0.3 * idf1 + 0.7 * idf2) * \
np.array([ner(t) * postag(t) for t in tks])
wts = [math.exp(s) for s in wts]
wts = [math.pow(s, 2) for s in wts]
tw = list(zip(tks, wts))
else:
for tk in tks:
@@ -237,7 +237,7 @@ class Dealer:
idf2 = np.array([idf(df(t), 1000000000) for t in tt])
wts = (0.3 * idf1 + 0.7 * idf2) * \
np.array([ner(t) * postag(t) for t in tt])
wts = [math.exp(s) for s in wts]
wts = [math.pow(s, 2) for s in wts]
tw.extend(zip(tt, wts))

S = np.sum([s for _, s in tw])

Chargement…
Annuler
Enregistrer