SLGP Header

Improving Labeling Quality using Positive Label Frequency Threshold Algorithm

IJCSEC Front Page
Author(s):
M.Kiruthiga, P.Sangeetha
Volume
4, Issue 6
Pages:
1467-1473
Year of Publication:
December,2016
International Journal of Computer Science and Engineering Communications
ISSN:
2347-8586
Citation: M.Kiruthiga, P.Sangeetha."Improving Labeling Quality using Positive Label Frequency Threshold Algorithm " International Journal of Computer Science and Engineering Communications,Vol.4,Issue.6(2016):1467-1473 BibTex

@article{2016MK14671473, author = {M,Kiruthiga. P,Sangeetha}, title = {Improving Labeling Quality using Positive Label Frequency Threshold Algorithm }, journal = {International Journal of Computer Science and Engineering Communications}, issue_date = {25}, volume = {4}, number = {6}, month = {Dec}, year = {2016}, issn = {2347-8586}, pages = {1467-1473}, numpages = {7}, url = {http://www.scientistlink.com/ijcsec/2016/V4I614671473.html}, publisher = {Scientist Link Group of Publications}, address = {Chennai, India} }

DOI: Full Text Download
Abstract:
Label is a prominent issue in the classification area along with several potential negative sequences. For example, the predicted accuracy may reduce, but the complexity of inferred models and the number of necessary training samples may rise. Online outsourcing systems, such as Amazon’s Mechanical Turk, allow labelers to label the same objects but still lack in their quality. Mostly noisy labels have multiple labels for same examples. Thus, an agnostic algorithm Positive LAbel frequency Threshold (PLAT) is projected to handle the issue of imbalanced noisy labeling. The main objective is to generate the training dataset and integrated labels of examples. This method is used to solve the issue of minority sample and also able to deal with imbalanced multiple noisy labeling. The PLAT is applied to the imbalanced dataset collected from Amazon Mechanical Turk and the experiment results represents that the PLAT is efficient than other methods.

Keywords:repeated labeling, majority voting, imbalanced labeling

References:

  1. C. L. Black and C. J. Merz. UCI repository of machine learning database [Online]. Available: http://archive.ics.uci.edu/ml/, 1998.
  2. N. V. Chawla, L. O. Hall, K. W. Bowyer, and W. P. Kegelmeyer, “SMOTE: Synthetic minority oversampling technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002.
  3. P. Donmez, J. G. Carbonell, and J. Schneider, “Efficiently learning the accuracy of labeling sources for selective sampling,” in Proc.15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2009, pp. 259–268.
  4. A. Estabrooks, T. Jo, and N. Japkowicz, “A multiple resampling method for learning from imbalanced data sets,” Comput. Intell., vol. 20, no. 1, pp. 18–36, 2004.
  5. H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, Sep. 2009.
  6. H. Kajino, Y. Tsuboi, and H. Kashima, “A convex formulation for learning from crowds,” in Proc. 26th AAAI Conf. Artif. Intell., 2012, pp. 73–79.
  7. A. Kumar and M. Lease, “Modeling annotator accuracies for supervised learning,” in Proc. 4th ACM WSDM Workshop Crowd sourcing Search Data Mining, 2011, pp. 19–22.
  8. X. Y. Liu, J. Wu, and Z. H. Zhou, “Exploratory under sampling for class imbalance learning,” in Proc. IEEE 6th Int. Conf. Data Mining, 2006, pp. 965–969.
  9. H. Y. Lo, J. C. Wang, H. M., Wang, and S. D., Lin, “Cost-sensitive multi-label learning for audio tag annotation and retrieval,” IEEE Trans. Multimedia, vol. 13, no. 3, pp. 518–529, Jun. 2011.
  10. C. Parker, “On measuring the performance of binary classifiers,” Knowl. Inform. Syst., vol. 35, no. 1, pp. 131–152, 2013.
  11. V. S. Sheng, “Simple multiple noisy label utilization strategies,” in Proc. IEEE 11th Int. Conf. Data Mining, 2011, pp. 635–644.
  12. V. S. Sheng, F. Provost, and P. Ipeirotis, “Get another label? Improving data quality and data mining using multiple, nosiy labeler,” in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2008, pp. 614–662.
  13. P. Smyth, M. C. Burl, U. M. Fayyad, P. Perona, and P. Baldi, “Inferring ground truth from subjective labeling of venus images,” Adv. Neural Inform. Process. Syst., vol. 8, pp. 1085–1092, 1995.
  14. R. Snow, B. O’Connor, D. Jurafsky, and A. Ng, “Cheap and fast— But is it good?” in Proc. Conf. Empirical Methods Natural Lang. Process., 2008, pp. 254–263.
  15. C. Strapparava and R. Mihalcea, “SemEval-2007 Task 14: Affective text,” in Proc. 4th Int. Workshop Semantic Eval., 2007, pp. 70–74.
  16. P. Welinder and P. Perona, “Online crowdsourcing: Rating annotators and obtaining cost-effective labels,” in Proc. Workshop Adv.Comput. Vis. Humans Loop, 2010, pp. 25–32.
  17. J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. Movellan, “Whose vote should count more: Optimal integration of labels from labelers of unknown expertise,” in Proc. Adv. Neural Info. Process. Syst. 22, 2009, pp. 2035–2043.
  18. J. Zhang, X. Wu, and Victor S. Sheng,”Imbalanced Multiple Noisy Labeling”, vol 27, feb 2015.