Sample code for Plagiarism Searcher tool

re: Sample code for Plagiarism Searcher tool

Mike Stall - MSFT — Wed, 24 Aug 2005 20:36:44 GMT

I apologize; I didn't intentd to accuse tagcloud of plagiarising - I'll update my original post to be very clear about that.
I explicitly called out tagcloud as an example of a false-postive:
"It turns out the 2% one is a false positive: the search engine cache found the copied content, but then the page had completely changed since then (it's an "Under Construction page" now)."

The problem is that the tool uses MSN Search (which uses a cache) to find candidates; but then live page access to look for credits to the author. It will search both the raw HTML and the stripped HTML. So having credits inside of tags (such as a <A>) will show up.

The problem is if the cached page has a credit, but the site is down (or replaced with a "I'm under construction" notice), the tool won't see it and report a false positive.
This would also be a problem for blog homepages where the entry that credits the author is no longer on the homepage.

This is a shortcoming of the tool. The tool needs to find candidates and check for crediting using the same copy of the HTML.

re: Sample code for Plagiarism Searcher tool

Grant — Wed, 24 Aug 2005 11:14:57 GMT

Hi Mike,

I was interested to see why your tool picked up the tagcloud.com URL for the tag "Readify"

Tagcloud shows extracts of blogs for particular tags, not unlike technorati.com.

Seeing as the site is down at the moment, this is an example of what your scanner would have seen (although it's been updated since)

http://cc.msnscache.com/cache.aspx?q=2145254345000&lang=en-US&FORM=CVRE

So your scanner would've seen the tagcloud of Readify, and of course Matthew's post.

It looks as though the summary strips the HTML so that's why there's no link back to your post.

Just to be safe, I've removed the Readify tagcloud so that it doesn't get accused of plagiarising again.

Grant

Interesting Finds

Jason Haley — Wed, 24 Aug 2005 08:12:12 GMT

re: Sample code for Plagiarism Searcher tool

Mike Stall - MSFT — Tue, 23 Aug 2005 17:33:04 GMT

I reran the tool and the Matthew's original page no longer shows up (since he added the credits). That's actually a great full-circle demo. I've updated the entry to reflect the new search.

re: Sample code for Plagiarism Searcher tool

Jonathan de Halleux — Tue, 23 Aug 2005 09:43:57 GMT

Michael,

Once you have extracted the "raw" data from MSN, you can apply this (http://research.microsoft.com/research/sv/PageTurner/similarity.htm) to get a better idead of the degree of plagiarism involved.

re: Sample code for Plagiarism Searcher tool

Mike Stall - MSFT — Tue, 23 Aug 2005 07:19:55 GMT

Matthew - No problem! I'm glad you found the content useful, and I had a lot of fun writing the searcher tool. :)

re: Sample code for Plagiarism Searcher tool

Matthew Cosier — Tue, 23 Aug 2005 04:59:49 GMT

Hi Mike,

Regarding your tools findings to the url above:
http://mcosier.blogspot.com/2005_06_01_mcosier_archive.html

If you take a look at the original post, you will see a white image (a box, representing the end of a quote). For some reason that I have only just realised, blogger doesn't actually provide a link to the original, but instead, it creates an image with an embeded reference to http://blogs.msdn.com/aggbug.aspx?PostID=POSTID

Unfortunately I didn't realise this until now when I was forced to open the HTML. I generally post using the blogger web client, and then leave it as that without checking the resultant view in a browser.

So perhaps either a) blogger needs to fix their stuff, or b) you should check for a reference to that url as well in your app...(not that I know anything about what aggbug.aspx does...)

Having said this, I will no longer rely on the blogger web client, and will never post with it again. As a matter of fact, I will probably start hosting my own very soon. I have fixed all references so that they are now direct links to your original.

I appologise for not checking my posts more thoughoughly... if you look at the post you will also see that I ended my part of the post with the colon:
I did not try and take credit for this, it was made clear that it wasnt my post.
I just wish blogger had of put a proper link in... gah.

Thanks,