ex Scientia : OpenXML

Ontology Add-in for Word 2007 - Technology Preview

pablofe — Wed, 11 Mar 2009 11:17:00 GMT

We just posted a Technology Preview build of an add-in that enables adding semantic knowledge to documents by associating words in the document to ontology terms. This add-in can be installed from CodePlex, and the source is available here under the Ms-PL license.

Add-in Basics

The add-in, developed in collaboration with the University of California San Diego and Science Commons, serves as a solution accelerator for those working in the ontology field. The add-in works in two ways:

- background scanning

- direct tagging

By default the add-in will scan terms in the document and present suggestions for terms it recognizes using SmartTags. Through the SmartTag menu, authors can associate recognized words to the appropriate ontology terms.

Authors can also tag words directly, by highlight the words and selecting the ontology term that they want to associate with them.

There are a large number of ontologies available online, using the OBO format, and through the configuration dialog, additional ontologies can be downloaded to the computer for use by the add-in.

Target Audience

Looking at the developer stack from higher to lower levels of abstraction, the add-in will be useful in at least three key areas:

· Development of new ontologies

· Investigation of new author interaction paradigms

· Integration into publishing and semantic workflows

For those developing new ontologies, the add-in provides a very easy way to test those ontologies with their target audience. In many scientific disciplines, Microsoft Word is a very popular tool for authoring papers and articles, and as such, authors are already familiar with its usage and features. The add-in is able to seamlessly build on this familiarity to expose new functionality and additional ontologies can be downloaded through a REST interface.

For researchers who are focused on new ways of analyzing content and detecting terms automatically, or on extending the user interaction with authors, access to the source code provides a great foundation to build on, without having to start from scratch. Also, community members can add incremental value to the add-in and share it back with others in the community. For this purpose, CodePlex provides a good forum to host discussions, report bugs, and publish documents related to the project.

Developers that work in the publishing industry, or at libraries and repositories, can customize or extend the add-in to present a user interface specific to their organization, or to add information to the XML content when terms are recognized and tagged by the add-in. Enhancing the information in the XML tag that captures the semantic information would also be useful to those doing semantic analysis, search, or storing information in databases.

Tagging

The add-in relies on custom XML tags to associate the semantic information with the matched words. The semantic information is stored as part of the document content. Utilities and applications that read or process docx files can retrieve and use this information (or transform it to other formats).

<w:customXml w:uri="http://biolit.ucsd.edu/biolitschema" w:element="biolit-term">

<w:customXmlPr>

<w:attr w:name="id" w:val="GO:0031386" />

<w:attr w:name="type" w:val="Biological process" />

<w:attr w:name="status" w:val="true" />

<w:attr w:name="OntName" w:val="Biological process" />

<w:attr w:name="url" w:val="http://purl.org/obo/owl/GO#GO_0031386" />

</w:customXmlPr>

<w:smartTag w:uri="BioLitTags" w:element="tag1">

<w:r>

<w:t>protein tag</w:t>

</w:r>

</w:smartTag>

</w:customXml>

Additional Functionality

The add-in also enables authors to search for terms in ontologies, look up their definition, as well as browse the ontologies to understand their organization and structure, as well as examine the terms. The add-in also provides a way to highlight tagged terms, which makes it simple to review the document and identify all tagged words.

A final useful piece of functionalty is the recognition of protein ID patterns from the National Center for Biotechnology Information (NCBI) and Protein Data Bank (PDB) databanks.

Beta of Microsoft’s Article Authoring Add-in Now Available Broadly for Download

pablofe — Fri, 30 May 2008 14:34:00 GMT

Enabling Journals to Better Connect with Scientific Authors in a Digital World

REDMOND, Wash. — May 30, 2008 — At the annual meeting of the Society for Scholarly Publishing in Boston, Microsoft announced the wide availability of the Beta 1 release of the Article Authoring Add-in for Microsoft Word 2007. In addition to enabling Word users to open and save documents using the National Library of Medicine’s XML Journal Publishing format, used for the authoring of scientific articles, the Beta 1 release adds support for the NCBI Book format, used for authoring book chapters for digital books.

Enabling Journals to Better Connect with Authors in a Digital World

A key value of the Article Authoring add-in is in enabling editors at scientific and technical journals to create article templates, tailored for their individual journals’ requirements. These templates will assist authors in writing articles with greater consistency in relation to the structure of the articles, better reflecting the content requirements of the journals, and in expressing semantic information which is key for the search and consumption of articles in digital form.

“The Add-In is a very positive development that will help scholars to write and tag their articles in the industry-standard NLM XML DTD, and will help publishers to process these articles in their editorial and production departments. We are pleased to be working with Microsoft on testing and refining this important tool that will benefit scholars and scholarly publishers alike”, said Ahmed Hindawi, CEO of Hindawi Publishing Corporation.

Preserving Information for Archiving and Search

The Article Authoring add-in enables authors to express a greater variety of semantic information, and metadata, as part of writing articles. This semantic information, captured in the XML format and preserved based on the extensibility in the Open XML standard, will prove valuable in improving the results from search queries and for the long term archival of scientific information.

In addition to preserving information that is native to Microsoft Word, the Beta 1 release of the Article Authoring add-in also preserves Math information from controls, such as Design Science’s MathType, when saving Word documents to the NLM XML format. Paul Topping, President and CEO of Design Science, Inc., stated that "We were happy to work with Microsoft to add support for Equation Editor and MathType equations to the Article Authoring add-in. Since at least 85% of the articles containing math submitted to scientific journals have equations in those formats, this support is critical."

The Open XML standard, with its capabilities to support custom-schemas, enables the Word add-in to support the entire set of rich information encoded by the NLM format. The add-in also provides easy access to the metadata in the NLM format, both by journal editors and by authors, directly from within the Word user interface. The broad availability of the Beta 1 release provides a way for the different communities, such as authors, journals, digital archives, and software vendors, to evaluate the technology and provide feedback, guiding further development of the add-in towards its initial release in the second half of 2008.

Information on how to download the Beta 1 release of the Article Authoring Add-in for Microsoft Office Word 2007 can be found at http://www.microsoft.com/downloads/details.aspx?FamilyID=09c55527-0759-4d6d-ae02-51e90131997e&displaylang=en.

The Power of Structured Content

pablofe — Wed, 14 May 2008 01:07:00 GMT

As we get comments and questions from you about the add-in, or have opportunities to discuss the experience and underlying technology face-to-face (as at the HighWire Publisher's Meeting last week), it is interesting how frequently the topic of structured content comes up.

Looking Back and Looking Forward

Although there is more awareness, visibility, and articles on the topic of legacy content (book scanning), it is important, and perhaps even more so, to focus on how new content is being created. As the consumption of content makes the transition from print to digital, and search becomes one of the key ways we come across new content (journals and conferences being the other traditional ways), it is important that we evolve the process by which new content is created, in order to be able to fully exploit the benefits of the new digital medium.

The way articles are commonly authored today is still largely focused on print as the end point, in that there is still a larger focus on presentation over semantics. The semantics in many workflows are added after the article is approved, somewhere along the publishing pipeline, by people other than the article authors. We need to enable authors to add semantics as part of the authoring process, and have that content preserved through the publishing workflow. The best technology available to preserve semantic content today is to express the article in XML. Archiving articles as plain text (either HTML or PDF) results in a loss of information. And, this loss of information is not only detrimental to effective search today, but will also be detrimental to other types of semantic analysis in the future.

Lossy Workflows

In some print focused workflows, even if semantic elements were present in the original file created by the author, the semantics are lost in the digital version, as a result of the process leading to print. At the end of this type of workflows, a PDF file is generated, reflecting the print layout. Any semantic information in the original article is lost in the resulting PDF. The PDF file, in turn, is used to generate an XML file, which is the basis for the digital content (and usually has to be sent out for tagging). These types of workflows not only result in a loss of semantic information, but are inefficient from the point of view of creating digital content. As journals move to be digital first (or digital only) editors will need to pay close attention to how their workflows are structured and how data is preserved/converted.

Presentation vs Semantics

Ideally, if we do a good job at capturing semantics during authoring, we can ignore the final presentation throughout the publishing workflow. Layout and other presentation elements (margins, font family, color, size, etc) can be applied to the archive version of the document for viewing (for example, during the generation of the HTML files). Of course, we do not want authors dealing with raw XML editing. Having a nice presentation within the word processor helps with authoring, but, it does not need to be the final presentation, instead it can be the presentation that best works for the author.

Currently some Word article templates rely on Styles to identify semantics. This is the best that could be done with the capabilities of previous versions of Word, but it is fragile as it mixes presentation and semantics. To make matters worse, it is easy for styles to "leak" through copy and paste, or editing, invalidating the semantics. Word 2007's use of XML as its native format, its ability to have custom XML elements, and the extensibility of Word's user interface and file packaging, enable a more robust way of entering semantic elements during authoring, preserving metadata, and enabling conversion to other formats.

Capturing Semantics and Authors' Insights

Authors will be largest class of users of the add-in, so we focus a lot on the experience that is presented to this audience. Authors likely will have no idea of the format used to back their articles (whether OpenXML or NLM), nor should they care. Also, the richness and complexity of the metadata expressible in the NLM format does not need to be exposed to them in a raw form (but needs to be accessible to the journal/archival staff - I will cover this in a future posting).

What is the benefit to authors from capturing semantics and metadata? As semantic search evolves, articles with more/better semantic data should become more relevant in search results than articles without this information. Thus, as content moves to be consumed primarily in digital form, articles with better semantics stand a better chance of being found, read, and cited.

Along the concept of the Dublin Core (or the core of the Core), we are focusing on enabling journals to capture a set of data from authors:

Sections (title, abstract, etc)
Authors information
Keywords and subjects
Author notes

While it would be interesting to capture additional information within the content during authoring, it is important not to overburden authors. At least if we manage to get this small set of data reliably, and reduce entry errors, then we will provide a good baseline for metadata in articles. Over time, as authors become comfortable with the concept, and see benefits, the baseline can be moved up (but again, the user interaction needs to be simple and, as much as possible, unobtrusive).

Additional Reading

Peter Murray-Rust has a couple of threads with related topics, on semantics and chemistry and structured content and PDF.

Multi Discipline Relevance

pablofe — Thu, 27 Mar 2008 07:59:00 GMT

Geoff makes the point that he thinks the NLM dtd is relevant for disciplines outside of Biology and Medicine, such as Humanities and Social Sciences. I agree with his point and hope that with the add-in we will help authors and journals in those other disciplines, which already tend to use Word, in the submission and conversion processes. Even going beyond the conversion to the NLM tagset, I think that there are other features in the add-in that will be of use to authors and to the editorial staff. I plan to cover both of these aspects in future postings.

Covering yet another set of scientific disciplines, last month the folks from ArXiv (the largest repository for Physics, also very popular for Math and Computer Science papers) posted the news that they now accept OpenXML files for submissions. I am also looking forward to being able to help scientists and researchers that deposit OpenXML files into ArXiv.

One fun aspect of this project is being able to work with the very sharp individuals who are the center of the ongoing evolution in the Scientific, Technical, and Medical publishing space. They range from folks in the repository space (from the National Library of Medicine, British Library, and ArXiv), in the staff of commercial publishers and journals, and key folks in many of the companies that develop tools for the publishing workflow.

Welcome to the Article Authoring Add-in

pablofe — Thu, 20 Mar 2008 10:55:00 GMT

Today we are making available a Technology Preview of the Article Authoring add-in for Word 2007, focused on the community of authors, editors, and publishers of scientific and technical articles. The goal is to simplify several activities in the publishing workflow, from authoring to publishing and archiving, with this last step including conversion to the XML format from the National Library of Medicine. The current process of getting an article from the authors to a journal (increasingly electronic only) is a bit complicated and many times lossy, especially in relation to the metadata related to the article, we hope that the add-in will help simplify and improve the process.

At the core of many publishing workflows is the XML format from the National Library of Medicine (the format is also used for long term archiving and preservation of articles – and actually there are four formats (DTDs) defined by the National Library of Medicine). Beyond the ability to save and open files in the NLM format from Word 2007, the add-in also enables editing of the metadata, which is an important part of the format, directly from within the Word user interface.

Additionally, the add-in provides the editorial staff at the different journals with the ability to define templates, used to assist authors in the writing process, so that, as an end result, articles submitted to journals more closely match the journal requirements in relation to the different sections allowed for the article, their length, and some of the metadata required for publishing.

Editing metadata is also an important aspect of the publishing workflow, the add-in enables authors and journal staff to access and edit metadata (supplementary information that complements the content of the article, and which is very useful for search) within the Word user interface.

Content and Metadata

Beyond the core content of the article provided by the authors, there is useful information that is attached to an article, which is important for search. The authors are best suited to enter some of this information, such as the author information (contact information, affiliation, biography), as well as some of the basic information about the content, such as keyword, and describing the taxonomy of the article (subjects being covered). In addition, there is metadata that the editorial staff needs to provide, such as the license for the article, publishing date, and informational about the journal where the article will be published.

This type of information is not accessible through the default user interface in Word, but the add-in extends the UI to enable editing of the metadata. The metadata entered is then kept with the content as part of the docx file, which should provide greater flexibility to the editorial and publishing staff, as they will be able to, for example, send files back to the authors to review last minute changes or to take updates.

OpenXML as the Enabling Technology

Starting from the OpenXML content simplifies the conversion process from one XML format to another, but beyond this, there are a couple of OpenXML features that make the overall solution come together. In future postings I will cover these in more detail, but custom schemas and the ability to store additional information in the file (through the Open Packaging Conventions) are key in being able to package the content and metadata in a single file, which can then be opened and edited by any tools as part of the publishing process.

The Technology Preview Build

We are providing this preview build to gather feedback and requirements. The target audience is the editorial and publishing staff at STM journals, companies that develop publishing tools, and technical staff at Information Repositories, libraries, and archives. Researchers and scientists that are early adopters can also download the preview to provide feedback on the user experience. Do not use this build for production purposes, only for evaluation.

We welcome your feedback and, if you run into any issues, please let us know about them as well. We will be posting updates and answering your questions in this blog. Also, let us know if there are specific topics you will like us to do follow up postings on.

We hope that you will find this add-in useful and that it will help simplify your workflow and authoring experience.

The NLM tagsets

The add-in provides support for the Journal Publishing DTD - http://dtd.nlm.nih.gov/publishing/tag-library/2.2/. For more information on the different formats, check out this overview http://dtd.nlm.nih.gov/.