Factorization Machines

Factorization Machines

{hirata-ai@ed, komachi@}tmu.ac.jp

1 Collins Named Entity Recognition

[4]

co-training

Primadhanty

[5]

SVD L1 L2 [1]

Factorization Machine

3 Factorization Machines


Factorization Machines [6] Support Vector Machine (SVM) (Matrix Factorization)

d=2

2 Random Fields CRF [2]

Factorization

Machines Conditional Support Vector Ma-

yˆ(x) := w0 +

chines SVM [3]

n !

w i xi +

i=1

1 i w ∈ Rn

1

n n ! !

i=1 j=i+1

n w

w0 ∈ R

⟨vi , vj ⟩xi xj

(1)

xi

x

2

wi

1

xi

3 k

vi , vj

2 k !

⟨vi , vj ⟩ := vi k

f =1

vi,f · vj,f

(2)

V ∈ Rn×k

i

2 Machines

1: Primadhanty

⟨·, ·⟩

PER

6,516 (3,489)

1040 (762)

1,342 (925)

LOC ORG MISC

6,159 ( 987) 5,721 (2,149) 3,205 ( 760)

176 (128) 400 (273) 177 (142)

246 (160) 638 (358) 213 (152)

O

36,673 (5,821)

951 (671)

995 (675)

Factorization

2

4.2 Markov Chain Stochastic Gradient De-

Monte Carlo (MCMC)

Primadhanty 12

scent (SGD) 2

V n×k

4.3

SVM Factorization Machines

scikit-learn

SVM Fac-

Version 0.17

torization Machine libFM Version 1.4.2 [7] SVM Factorization Machines one-vsFactorization

Factorization Machines all Machines

d=2 k

4 4.4 precision

Primadhanty

recall

F1

O 4

SVM Factorization Machine

4.5

4.1

3

Primadhanty Primadhanty

CoNLL-2003

1

Primadhanty 4

1

precision-recall Factorization Machines

PER ORG O

LOC

Primadhanty precision

MISC

recall

1

1 F1

1

Factories 1

1 Primadhanty

POS

2: cap=1, cap=0 all-low=1, all-low=0 all-cap1=1, all-cap1=0 all-cap2=1, all-cap2=0 num-tokens=1, num-tokens=2, num-tokens>2

1

2

dummy Primadhanty

k

3: Primadhanty

k=5 precision

recall

F1

49.75

44.50

46.75

53.75 62.03

50.67 53.92

51.94 55.88

60.93

55.10

57.27

SVM [5] Factorization Machine

F1 k=1

57.1 k=5

k=8 F1

k k=5

Primadhanty

40 Factorization Machines

Machines

10 Factorization Machines Primadhanty

5 “ORGANIZATION”

40


“ORGANIZATION” “Vice-

“OTHER”

Primadhanty

President”


“Vice-President”

2 SVD Factorization Machines

“LOCATION” “PERSON” 2

6 PERSON

Factorization Machines Primadhanty Factorization Machines Factorization

Machines Primadhanty 2


4: PERSON SVM [5] Factorization Machine

LOCATION

MISC

P

R

F1

P

R

F1

P

R

F1

P

R

F1

86.45

72.28

78.73

31.35

38.62

34.61

62.54

59.40

60.93

34.67

32.39

33.50

73.83 84.36

90.84 80.40

81.46 82.33

64.96 39.49

36.18 50.41

46.48 44.29

72.11 70.88

44.98 55.33

55.41 62.15

37.20 48.99

43.66 34.27

40.17 40.33

1:

.

ORGANIZATION

[1]

,

[2]

. Semi-Markov conditional random fields . , , 2006.

[3]

,

precision-recall

. , Vol. 49, No. 11, pp. 3765–3776, 2008.

, .

. Support vector machine , Vol. 43, No. 1,

pp. 44–53, 2002. [4] Michael Collins and Yoram Singer. Unsupervised models for named entity classification. In EMNLP, pp. 100–110, 1999. [5] Audi Primadhanty and Xavier Carreras Ariadna Quattoni. Low-rank regularization for sparse conjunctive feature spaces: An application to named entity classification. In Proceedings of ACL-IJCNLP, pp. 126–135, 2015. [6] Steﬀen Rendle. Factorization machines. In Proceedings of ICDM, pp. 995–1000, 2010. [7] Steﬀen Rendle. Factorization machines with libFM. ACM Trans. Intell. Syst. Technol., Vol. 3, No. 3, pp. 57:1–57:22, 2012.

2: Factorization Machines

k