Our Named Entity Recognition (NER) system employs Graph-Based-Decoding (GBD) over a hybrid network architecture composed of bidirectional LSTMs for word-level encoding, with a MultiLayerPerceptron for detecting links inside entities and a unidirectional LSTM for label estimation:

×

We are currently working on integrating the NER branch into master. Until then, you can check out the source for the GDB-NER system at this repository

Note: During the official PARSEME evaluation our system (GBD-NER) was affected by a bug which caused it to use only delexicalized features as input. After correcting this bug, we have re-trained and evaluated our system. See below for details:

Cross-lingual macro-averages

General ranking

System	Track	#Langs	MWE-based				Token-based
			P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	19/19	60.36	53.38	56.65	1	66.28	55.78	60.57	1
TRAVERSAL	closed	19/19	67.58	44.97	54	2	77.41	48.55	59.67	2
TRAPACC_S	closed	19/19	62.28	41.4	49.74	3	68.54	42.06	52.13	5
TRAPACC	closed	19/19	55.68	44.67	49.57	4	62.1	46.37	53.09	4
CRF-Seq-nocategs	closed	19/19	56.13	39.12	46.11	5	73.44	43.49	54.63	3
varIDE	closed	19/19	61.49	36.71	45.97	6	64.13	37.63	47.43	7
CRF-DepTree-categs	closed	19/19	52.33	37.83	43.91	7	64.65	41.56	50.6	6
GBD-NER-standard	closed	19/19	36.56	48.3	41.62	8	41.11	52.21	46	8
GBD-NER-resplit	closed	19/19	30.26	52.95	38.51	9	33.83	58.03	42.74	10
Veyn	closed	19/19	42.76	32.51	36.94	10	58.13	36.57	44.9	9
mumpitz	closed	19/7	17.14	13.03	14.81	11	24.95	15.5	19.12	12
Polirem-rich	closed	19/3	10.9	2.87	4.54	12	13.07	3.89	6	13
Polirem-basic	closed	19/3	10.78	0.65	1.23	13	11.33	0.68	1.28	14
MWETreeC	closed	19/19	0.21	3.72	0.4	14	23.5	24.78	24.12	11
SHOMA	open	19/19	66.08	51.82	58.09	1	76.22	54.27	63.4	1
Deep-BGT	open	19/10	33.41	25.29	28.79	2	39.77	26.47	31.78	2
Milos	open	19/4	9.17	7.87	8.47	3	11.5	8.25	9.61	3
mumpitz-preinit	open	19/1	2.28	1.9	2.07	4	3.71	2.35	2.88	4

↑ Back to top

Language-specific system rankings

BG

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	79.41	56.42	65.97	1	82.10	55.72	66.39	1
varIDE	closed	63.58	61.49	62.52	2	66.82	61.16	63.86	2
TRAPACC	closed	70.61	53.43	60.83	3	72.94	54.45	62.35	3
CRF-Seq-nocategs	closed	76.3	48.06	58.97	4	82.67	49.51	61.93	6
TRAVERSAL	closed	75.59	47.61	58.42	5	82.36	49.79	62.06	4
GBD-NER-standard	closed	50.49	69.1	58.35	6	52.94	70.48	60.47	7
mumpitz	closed	75.12	46.42	57.38	7	86.99	48.16	62	4
CRF-DepTree-categs	closed	73.92	41.04	52.78	8	77.9	41.31	53.99	8
TRAPACC_S	closed	80.8	38.96	52.57	9	84.45	39.12	53.47	9
GBD-NER-resplit	closed	39.01	71.79	50.55	10	41.23	73.87	52.92	10
Veyn	closed	0	0	0	n/a	0	0	0	n/a
MWETreeC	closed	0	0	0	n/a	5.05	3.04	3.79	11
Deep-BGT	open	85.96	52.99	65.56	1	91	52.82	66.85	1
SHOMA	open	75.6	55.97	64.32	2	79.98	56.43	66.17	2

↑ Back to top

DE

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	53.26	39.36	45.27	1	61.13	42.26	49.97	4
TRAPACC	closed	53.28	37.55	44.05	2	62.4	39.49	48.37	5
TRAVERSAL	closed	62.93	32.73	43.06	3	76.17	38.36	51.02	3
GDB-NER-full	closed	44.13	33.94	38.37	4	64.24	43.49	51.87	1
CRF-DepTree-categs	closed	52.71	29.32	37.68	5	61.87	33.95	43.84	7
mumpitz	closed	32.15	38.35	34.98	6	55.91	48	51.66	2
CRF-Seq-nocategs	closed	40.21	23.09	29.34	7	69.68	33.95	45.66	6
GBD-NER-resplit	closed	27.16	29.72	28.38	8	34.33	41.33	37.51	10
GBD-NER-standard	closed	26.43	27.91	27.15	9	35.97	40.62	38.15	9
Veyn	closed	32.65	22.29	26.49	10	57.52	31.38	40.61	8
varIDE	closed	82.35	8.43	15.3	11	82.35	5.74	10.74	11
MWETreeC	closed	0.76	2.81	1.2	12	4.07	7.69	5.32	12
Deep-BGT	open	60.94	36.35	45.53	1	77.92	37.64	50.76	3
Milos	open	63.74	34.94	45.14	2	67.71	24.51	35.99	4
SHOMA	open	54.15	37.95	44.63	3	69.7	40.82	51.49	2
mumpitz-preinit	open	43.37	36.14	39.43	4	70.5	44.62	54.65	1

↑ Back to top

EL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	64.74	49.10	55.85	1	74.02	52.22	61.24	1
TRAPACC_S	closed	61.65	41.72	49.76	2	70.25	42.68	53.1	4
TRAVERSAL	closed	65.7	36.33	46.79	3	82.16	42.01	55.59	2
TRAPACC	closed	64.31	36.33	46.43	4	72.86	37.07	49.14	6
CRF-DepTree-categs	closed	50.8	38.12	43.56	5	61.44	41.34	49.42	5
CRF-Seq-nocategs	closed	54.41	35.73	43.13	6	74.66	41.92	53.7	3
GBD-NER-resplit	closed	29.55	67.07	41.03	7	36.12	74.56	48.66	8
GBD-NER-standard	closed	41.68	39.52	40.57	8	51.95	44.52	47.95	9
mumpitz	closed	45	30.54	36.39	9	73.21	36.82	49	7
varIDE	closed	86.51	21.76	34.77	10	87.73	20.33	33.02	11
Veyn	closed	39.01	21.96	28.1	11	70.02	28.54	40.55	10
MWETreeC	closed	0	0	0	n/a	41.13	24.44	30.66	12
SHOMA	open	62.79	53.89	58	1	78.6	58.08	66.79	1
Milos	open	42.76	37.72	40.08	2	64.16	47.03	54.27	2

↑ Back to top

EN

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC	closed	38.4	28.74	32.88	1	42.23	28.98	34.37	1
TRAVERSAL	closed	55.5	21.16	30.64	2	58.31	20.33	30.15	3
TRAPACC_S	closed	49.77	21.76	30.28	3	53.5	21.07	30.23	2
CRF-Seq-nocategs	closed	49.76	20.36	28.9	4	55.95	20.33	29.82	4
CRF-DepTree-categs	closed	40.87	18.76	25.72	5	47.2	18.58	26.67	5
varIDE	closed	59.38	15.17	24.17	6	61.09	14.44	23.36	6
GBD-NER-standard	closed	13.69	32.53	19.27	7	14.94	32.84	20.54	7
GDB-NER-full	closed	10.52	57.68	17.79	8	11.86	60.81	19.85	8
GBD-NER-resplit	closed	9.55	42.32	15.59	9	11	45.17	17.69	9
Veyn	closed	27.78	2.99	5.41	10	40.66	3.4	6.28	11
MWETreeC	closed	0.61	0.6	0.6	11	22.4	10.12	13.94	10
Milos	open	33.81	32.73	33.27	1	37.32	31.83	34.36	1
SHOMA	open	45.67	11.58	18.47	2	56.09	11.87	19.59	2

↑ Back to top

ES

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	29.54	40	33.98	1	35.96	44.43	39.75	2
CRF-Seq-nocategs	closed	30.87	36	33.24	2	37.47	41.74	39.49	3
TRAPACC	closed	25.69	41.2	31.64	3	32.04	46.78	38.03	4
TRAVERSAL	closed	28.84	33.4	30.95	4	39.91	40.26	40.09	1
GDB-NER-full	closed	22.22	36.80	27.71	5	30.38	46.17	36.65	7
varIDE	closed	17.02	49.2	25.3	6	20.15	53.74	29.31	8
CRF-DepTree-categs	closed	15.79	28.2	20.24	7	26.59	35.22	30.3	5
GBD-NER-standard	closed	11.83	44.6	18.7	8	15.64	53.48	24.2	10
Veyn	closed	17.28	16.8	17.04	9	31.68	23.83	27.2	9
GBD-NER-resplit	closed	9.1	42.8	15.01	10	12.4	53.48	20.13	12
mumpitz	closed	9.66	13	11.08	11	31.83	28.87	30.28	6
MWETreeC	closed	0	0	0	n/a	31.03	17.57	22.43	11
SHOMA	open	31.65	48.8	38.39	1	38.33	53.57	44.69	1
Deep-BGT	open	24.5	34.2	28.55	2	33.13	38.61	35.66	2

↑ Back to top

EU

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	82.07	71.40	76.36	1	83.12	71.79	77.14	1
TRAPACC_S	closed	84.69	68.6	75.8	2	86.07	69.38	76.83	2
TRAPACC	closed	85.56	64	73.23	3	86.97	65.01	74.4	4
CRF-Seq-nocategs	closed	77.67	62.6	69.32	4	86.42	65.81	74.72	3
TRAVERSAL	closed	78.28	58.4	66.9	5	83.42	65.01	73.07	5
CRF-DepTree-categs	closed	69.29	58.2	63.26	6	77.04	61.03	68.11	6
Veyn	closed	60.15	64	62.02	7	66.6	68.19	67.39	7
varIDE	closed	39.87	76	52.31	8	41.61	78.83	54.46	8
GBD-NER-standard	closed	45.05	58.2	50.79	9	45.67	58.65	51.35	9
GBD-NER-resplit	closed	29.18	69.8	41.16	10	29.97	71.27	42.2	10
MWETreeC	closed	0	0	0	n/a	3.07	4.67	3.71	11
SHOMA	open	81.8	72.8	77.04	1	86.27	74.95	80.21	1

↑ Back to top

FA

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	81.96	75.25	78.46	1	88.60	77.54	82.70	1
GBD-NER-resplit	closed	78.23	77.45	77.83	2	84.13	78.62	81.28	2
GBD-NER-standard	closed	78.11	74.05	76.02	3	84.54	75.65	79.85	3
TRAPACC	closed	86.98	66.67	75.48	4	92.07	67.83	78.12	5
TRAPACC_S	closed	87.1	64.67	74.23	5	93.12	65.68	77.03	6
CRF-Seq-nocategs	closed	81.22	63.87	71.51	6	93.05	67.39	78.17	4
CRF-DepTree-categs	closed	77.69	57.68	66.21	7	87.98	59.84	71.23	8
TRAVERSAL	closed	73.8	58.48	65.26	8	90.19	65.23	75.7	7
Veyn	closed	80.44	50.9	62.35	9	92.47	54.09	68.25	9
varIDE	closed	91.98	29.74	44.95	10	96.6	28.12	43.56	10
MWETreeC	closed	0	0	0	n/a	55.49	35.85	43.56	10
SHOMA	open	86.12	71.86	78.35	1	93.87	74.3	82.95	1

↑ Back to top

FR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	77.19	44.18	56.19	1	84.72	48.76	61.9	1
GDB-NER-full	closed	72.76	42.37	53.55	2	81.85	44.66	57.79	2
varIDE	closed	55.24	46.59	50.54	3	63.4	49.7	55.72	3
TRAPACC	closed	61.09	38.15	46.97	4	70.55	42.36	52.93	5
CRF-DepTree-categs	closed	61.17	37.95	46.84	5	68.15	40.56	50.86	7
CRF-Seq-nocategs	closed	63.6	36.14	46.09	6	82.66	41.93	55.64	4
TRAPACC_S	closed	72.1	33.73	45.96	7	82.26	36.04	50.12	8
mumpitz	closed	56.8	33.53	42.17	8	81.25	38.86	52.57	6
GBD-NER-standard	closed	36.93	34.34	35.59	9	46.02	38.51	41.93	10
Veyn	closed	39.95	31.12	34.99	10	62.26	39.45	48.3	9
GBD-NER-resplit	closed	22.14	44.38	29.55	11	27.62	51.15	35.87	11
Polirem-rich	closed	68.82	12.85	21.66	12	82.59	17.42	28.77	13
Polirem-basic	closed	78.05	6.43	11.87	13	83.51	6.92	12.78	14
MWETreeC	closed	0	0	0	n/a	34.68	26.47	30.02	12
SHOMA	open	71.86	52.81	60.88	1	79.04	56.02	65.57	2
Deep-BGT	open	57.81	49.8	53.51	2	78.88	56.45	65.8	1
Milos	open	34	44.18	38.43	3	49.29	53.46	51.29	3

↑ Back to top

HE

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	50.33	15.14	23.28	1	74.64	18.1	29.13	2
GBD-NER-standard	closed	15.89	39.64	22.69	2	19.83	44.94	27.52	3
TRAPACC	closed	36.41	14.14	20.37	3	47.59	16.28	24.26	4
Veyn	closed	23.39	17.33	19.91	4	39.29	23.81	29.65	1
varIDE	closed	69.23	10.76	18.62	5	70.3	10.04	17.58	8
TRAPACC_S	closed	56.82	9.96	16.95	6	69.02	11	18.97	6
GDB-NER-full	closed	74.19	09.16	16.31	7	75.76	08.66	15.54	10
GBD-NER-resplit	closed	8.55	48.21	14.52	8	10.8	56.1	18.11	7
CRF-DepTree-categs	closed	22.43	9.56	13.41	9	37.82	14.11	20.55	5
MWETreeC	closed	0	0	0	n/a	36.62	9.96	15.66	9
CRF-Seq-nocategs	closed	0	0	0	n/a	26.98	1.47	2.79	11
SHOMA	open	61.37	28.49	38.91	1	74.43	31.26	44.02	1

↑ Back to top

HI

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
CRF-DepTree-categs	closed	77.95	68.6	72.98	1	81.12	66.94	73.35	1
CRF-Seq-nocategs	closed	76.79	64.2	69.93	2	81.8	64.31	72.01	2
TRAPACC	closed	72.39	66.6	69.38	3	77.17	66.12	71.22	3
TRAPACC_S	closed	82.91	58.2	68.39	4	86.53	56.43	68.31	5
GDB-NER-full	closed	61.90	72.80	66.91	5	65.18	71.38	68.14	6
TRAVERSAL	closed	66.3	60.6	63.32	6	73.15	67.12	70	4
GBD-NER-standard	closed	52.97	74.8	62.02	7	55.95	73.28	63.45	7
GBD-NER-resplit	closed	52.97	74.8	62.02	7	55.95	73.28	63.45	7
varIDE	closed	85.2	42.6	56.8	8	86.77	41.58	56.22	8
Veyn	closed	0	0	0	n/a	0	0	0	n/a
MWETreeC	closed	0	0	0	n/a	44.82	29.8	35.8	9
SHOMA	open	76.84	69	72.71	1	84.3	68.57	75.62	1

↑ Back to top

HR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	68.04	46.59	55.3	1	78.14	50.73	61.52	1
TRAPACC_S	closed	60.42	34.94	44.27	2	68.42	36.9	47.95	5
TRAPACC	closed	51.37	37.55	43.39	3	61.42	42.12	49.97	3
GDB-NER-full	closed	51.52	37.35	43.31	4	64.41	42.22	50.69	2
GBD-NER-resplit	closed	36.69	44.58	40.25	5	45.29	51.1	48.02	4
GBD-NER-standard	closed	36.53	43.57	39.74	6	44.02	48.53	46.17	7
CRF-DepTree-categs	closed	40.13	24.1	30.11	7	65.93	32.97	43.96	8
CRF-Seq-nocategs	closed	36.98	23.09	28.43	8	75.05	34.71	47.46	6
Veyn	closed	34.07	18.67	24.12	9	71.67	27.11	39.34	9
varIDE	closed	79.07	6.83	12.57	10	83.7	7.05	13.01	11
MWETreeC	closed	0	0	0	n/a	55.52	15.66	24.43	10
SHOMA	open	59.58	39.96	47.84	1	79.52	45.88	58.19	1

↑ Back to top

HU

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC	closed	92.44	88.27	90.31	1	90.72	85.44	88	1
TRAPACC_S	closed	94.87	85.82	90.12	2	93.56	81.75	87.25	2
CRF-Seq-nocategs	closed	93.94	77.96	85.21	3	95.16	76.31	84.7	3
TRAVERSAL	closed	88.01	74.74	80.84	4	89.91	79.61	84.45	4
GDB-NER-full	closed	85.47	76.55	80.76	5	90.54	77.09	83.27	6
CRF-DepTree-categs	closed	88	73.71	80.22	6	90.2	73.3	80.88	7
Veyn	closed	83.55	75.26	79.19	7	93.81	75.05	83.39	5
GBD-NER-resplit	closed	67.52	20.36	31.29	8	69.66	31.65	43.52	8
GBD-NER-standard	closed	67.12	19.2	29.86	9	68.92	29.71	41.52	9
varIDE	closed	100	10.31	18.69	10	100	8.74	16.07	10
MWETreeC	closed	2.58	67.14	4.96	11	3.81	74.85	7.25	11
SHOMA	open	90.08	81.96	85.83	1	93.49	80.87	86.73	1
Deep-BGT	open	78	71.26	74.48	2	80.71	73.11	76.72	2

↑ Back to top

IT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	63.09	40.32	49.2	1	74.42	42.11	53.78	1
GDB-NER-full	closed	49.52	41.73	45.30	2	62.46	45.59	52.71	2
TRAPACC	closed	52.43	30.44	38.52	3	61.54	30.34	40.64	5
CRF-Seq-nocategs	closed	55.14	27.02	36.27	4	78.49	33.05	46.51	3
TRAPACC_S	closed	55.66	23.79	33.33	5	65.42	22.99	34.02	9
CRF-DepTree-categs	closed	44.76	25.81	32.74	6	58.78	29.8	39.55	6
varIDE	closed	31.07	34.07	32.5	7	39.22	35.06	37.02	7
Veyn	closed	34.01	30.44	32.13	8	58.41	38.16	46.16	4
Polirem-rich	closed	72.36	17.94	28.76	9	86.54	21.9	34.96	8
GBD-NER-standard	closed	15.45	29.84	20.36	10	22.68	35.45	27.67	10
GBD-NER-resplit	closed	10.69	28.83	15.59	11	16.63	38.31	23.19	11
Polirem-basic	closed	83.33	4.03	7.69	12	81.82	3.48	6.68	12
MWETreeC	closed	0	0	0	n/a	1.45	6.58	2.38	13
SHOMA	open	50.37	41.33	45.4	1	67.49	46.59	55.13	1
Deep-BGT	open	45.52	25.6	32.77	2	70	27.63	39.62	2

↑ Back to top

LT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	45.76	24.8	32.17	1	53.52	25.02	34.1	2
TRAPACC	closed	33.98	28.2	30.82	2	42.48	28.94	34.43	1
TRAVERSAL	closed	29.61	13.8	18.83	3	55.56	16.92	25.94	3
GDB-NER-full	closed	07.11	44.80	12.27	4	08.63	49.60	14.70	4
Veyn	closed	18.89	6.8	10	5	48.85	11.31	18.37	5
GBD-NER-standard	closed	3.76	39.6	6.87	6	4.76	45.95	8.62	6
GBD-NER-resplit	closed	3.76	39.6	6.87	7	4.76	45.95	8.62	6
varIDE	closed	55.56	1	1.96	7	55.56	1.34	2.61	8
MWETreeC	closed	0	0	0	n/a	0.42	44.52	0.84	9
CRF-Seq-nocategs	closed	0	0	0	n/a	35.16	2.85	5.27	7
CRF-DepTree-categs	closed	0	0	0	n/a	35.16	2.85	5.27	7
SHOMA	open	35.74	16.8	22.86	1	56.76	18.7	28.13	1

↑ Back to top

PL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	75.06	62.52	68.22	1	78.30	62.18	69.32	1
TRAVERSAL	closed	77.02	59.22	66.96	2	81.85	59.03	68.59	2
TRAPACC	closed	71.62	52.43	60.54	3	78.01	53.79	63.68	4
GBD-NER-standard	closed	59.36	60.97	60.15	4	63.93	61.73	62.81	6
TRAPACC_S	closed	78.55	48.35	59.86	5	83.1	48.83	61.51	7
Veyn	closed	62.69	55.15	58.68	6	73.52	58.12	64.92	3
CRF-Seq-nocategs	closed	70.88	46.8	56.37	7	83.28	50.81	63.12	5
GBD-NER-resplit	closed	42.53	67.96	52.32	8	45.77	69.4	55.16	8
mumpitz	closed	62.07	38.45	47.48	9	80.92	41.34	54.72	9
CRF-DepTree-categs	closed	26.66	44.47	33.33	10	37.99	57.94	45.89	10
varIDE	closed	86.11	6.02	11.25	11	88.89	5.78	10.85	12
MWETreeC	closed	0	0	0	n/a	43.78	30.51	35.96	11
SHOMA	open	73.05	56.31	63.6	1	79.32	57.13	66.42	2
Deep-BGT	open	70.87	56.7	63	2	80.23	57.85	67.23	1

↑ Back to top

PT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	76.8	52.08	62.07	1	85.14	54.69	66.6	1
varIDE	closed	72.8	52.26	60.84	2	76.1	52.85	62.38	3
GDB-NER-full	closed	66.59	55.88	60.77	3	69.78	57.58	63.09	2
Veyn	closed	62.55	52.26	56.95	4	71.26	52.69	60.58	4
TRAPACC	closed	55.01	50.63	52.73	5	61.55	53.01	56.96	6
TRAPACC_S	closed	65.75	43.4	52.29	6	72.63	43.62	54.51	8
CRF-DepTree-categs	closed	55.1	41.05	47.05	7	63.23	45.23	52.73	9
mumpitz	closed	44.77	47.2	45.95	8	63.96	52.37	57.58	5
CRF-Seq-nocategs	closed	48.51	35.26	40.84	9	78.01	44.67	56.81	7
Polirem-rich	closed	65.83	23.69	34.84	10	79.23	34.56	48.13	10
GBD-NER-standard	closed	21.08	49.37	29.55	11	23.79	51.24	32.49	11
GBD-NER-resplit	closed	14.83	63.29	24.03	12	16.4	66.96	26.34	12
Polirem-basic	closed	43.48	1.81	3.47	13	50	2.57	4.88	13
MWETreeC	closed	0	0	0	n/a	0.94	44.35	1.85	14
SHOMA	open	71.12	65.46	68.17	1	78.81	68.89	73.51	1
Deep-BGT	open	72.44	46.11	56.35	2	79.4	44.83	57.3	2

↑ Back to top

RO

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	88.35	83.70	85.96	1	89.52	83.52	86.41	1
TRAPACC	closed	84.5	86.08	85.28	2	85.19	86.2	85.69	2
TRAPACC_S	closed	89.17	78.27	83.36	3	90.05	77.76	83.45	6
TRAVERSAL	closed	86.06	79.63	82.72	4	88.84	82.26	85.42	3
Veyn	closed	80.03	83.7	81.83	5	82.79	84.23	83.5	5
CRF-Seq-nocategs	closed	83.45	79.63	81.49	6	87.78	80.99	84.25	4
varIDE	closed	58.91	89.81	71.15	7	60.24	89.98	72.17	7
CRF-DepTree-categs	closed	74.69	62.14	67.84	8	81.11	63.64	71.32	8
GBD-NER-standard	closed	39.28	85.57	53.85	9	41.22	86.99	55.93	9
GBD-NER-resplit	closed	28.12	87.95	42.62	10	28.85	89.04	43.57	10
MWETreeC	closed	0	0	0	n/a	57.6	31.39	40.63	11
SHOMA	open	87.78	86.59	87.18	1	90.21	87.22	88.69	1
Deep-BGT	open	79.8	69.1	74.07	2	92.11	73.66	81.86	2

↑ Back to top

SL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	79.41	54	64.29	1	83.61	54.54	66.01	1
GDB-NER-full	closed	61.07	52.40	56.40	2	68.57	55.08	61.09	2
varIDE	closed	30.31	70.2	42.34	3	32.64	71.61	44.84	7
GBD-NER-standard	closed	34.56	48.8	40.46	4	40.34	51.39	45.2	6
Veyn	closed	57.84	31	40.36	5	72.87	33.06	45.49	5
CRF-Seq-nocategs	closed	55.67	31.4	40.15	6	82.14	38.01	51.97	3
CRF-DepTree-categs	closed	54.51	30.2	38.87	7	84.21	37.38	51.77	4
TRAPACC_S	closed	33.33	29.6	31.36	8	47.24	32.26	38.33	8
GBD-NER-resplit	closed	21.16	59.2	31.17	9	24.49	64.24	35.46	9
TRAPACC	closed	20.21	26.8	23.04	10	36.99	32.7	34.72	10
MWETreeC	closed	0	0.2	0	n/a	1.23	44.83	2.4	11
SHOMA	open	58.56	47.2	52.27	1	74.81	52.29	61.55	1
Deep-BGT	open	58.9	38.4	46.49	2	72.19	40.34	51.76	2

↑ Back to top

TR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GDB-NER-full	closed	68.07	54.35	60.44	1	70.06	54.64	61.40	1
GBD-NER-standard	closed	44.47	46.05	45.24	2	47.9	48.04	47.97	5
Veyn	closed	58.07	36.96	45.17	3	70.77	42.39	53.02	2
CRF-Seq-nocategs	closed	71.05	32.02	44.14	4	89.04	36.56	51.83	3
CRF-DepTree-categs	closed	67.71	29.84	41.43	5	84.62	33.68	48.19	4
TRAVERSAL	closed	81.48	26.09	39.52	6	88.38	27.66	42.13	6
GBD-NER-resplit	closed	44.11	25.89	32.63	7	47.4	27.08	34.47	7
varIDE	closed	4.19	65.22	7.87	8	5.23	78.95	9.81	8
TRAPACC	closed	1.64	1.58	1.61	9	5.26	4.21	4.68	10
TRAPACC_S	closed	1.12	0.59	0.78	10	6	2.01	3.01	11
MWETreeC	closed	0	0	0	n/a	3.35	8.52	4.81	9
SHOMA	open	81.4	45.85	58.66	1	87.22	47.66	61.63	1