originally by lingpipe
Here’s a 2 page write-up of one of the models I’ve been looking at for evaluating data annotation in order to evaluate coding standards and annotator sensitivity and specificity:
* Carpenter, Bob. 2008. Hierarchical Bayesian Models of Categorical Data Analysis.
I’ve submitted it as a poster to the New York Academy of Sciences 3rd Annual Machine Learning Symposium, which will be October 10, 2008.
Please let me know what you think (carp@alias-i.com). I didn’t have room to squeeze in the more complex model that accounts for “easy” items. This model and the one for easy items derive from the epidemiology literature (cited in the paper), where they’re trying to estimate disease prevalence from a heterogeneous set of tests. I’ve added some more general Bayesian reasoning, and suggested applications for annotation (though Bruce and Wiebe were mostly there in their 1999 paper, which I cite), and for training using probabilistic supervision (don’t think anyone’s done this yet).
I’m happy to share the R scripts and BUGS models I used to generate the data, fit the models, and display the results. I’d also love to know how to get rid of those useless vertical axes in the posterior histogram plots.
Hierarchical Bayesian Models of Categorical Data Annotation
Posted by jeffy
Posted on 8:20 PM
with No comments
loading..
Popular Posts
-
Resources about lucene Resources Introductions The API documentation contains a short and simple code example that show...
-
Just read a post from http://blog.bigml.com/2013/02/21/everything-you-wanted-to-know-about-machine-learning-but-were-too-afraid-to-ask-pa...
-
From https://de.dariah.eu/tatom/preprocessing.html Also refer to http://www.nltk.org/api/nltk.tokenize.html#module-nltk.tokenize ...
-
We examine top Python Machine learning open source projects on Github, both in terms of contributors and commits, and identify most popula...
-
Repost from http://terrytao.wordpress.com/advice-on-writing-papers/ There are three rules for writing the novel. Unfortunately, no on...
-
This plugin can be used to read a RSS feed and transform it into a custom piece of HTML. Setup <!DOCTYPE html> <html> <...
-
This is a very useful list of surveys from Doug Oard , which would be definitely helpful for those who want to enter this territory. I...
-
Teach Yourself Programming in Ten Years Peter Norvig Why is everyone in such a rush? Walk into any bookstore, and you’ll see how to Te...
-
file的这几个取得path的方法各有不同,下边说说详细的区别 概念上的区别:(内容来自jdk,个人感觉这个描述信息,只能让明白的人明白,不明白的人看起来还是有点难度(特别试中文版,英文版稍好些)所以在概念之后我会举例说明。如果感觉看概念很累就跳过直接看例子吧。看完例子回...
-
Data Science in Python by Greg http://blog.yhathq.com/posts/data-science-in-python-tutorial.html?from=timeline&isappinstalled=0 ...
0 Comments:
Post a Comment