For this article, we investigate how Hemingway Editor, an editing app, compares with a human editor in improving the overall quality of a writing sample and removing any mistakes. Keep reading to see how well each performed and where they fell short.
While Hemingway Editor had several positive features, it failed to accomplish the goal: to improve the writing sample and remove mistakes in grammar, punctuation, and spelling.
The human editor outperformed Hemingway Editor because they actually made changes to the writing sample, removed all writing mistakes, and greatly improved its overall quality and readability.
Hemingway Editor is an AI writing assistant that highlights words, phrases, and sentences in a text, showing you what areas need improvement. It looks at five common writing errors:
Each passage is given a “readability grade,” which ranges from 5 to 15. The readability grade indicates the reading level required to fully understand the given text. The Hemingway Editor suggests that users aim to achieve a readability level of 9 for most texts.
The point of the readability grades is to point out words, sentences, or paragraphs that may be too lengthy or filled with jargon or that may be confusing or tedious for your readers – all of which are undesirable qualities for any text.
The writing sample used for this experiment was a passage from a post-doctorate student’s thesis (used with permission). The author’s first language is not English, but they have studied English at a high level throughout their higher education. The writing sample was 605 words in length.
We submitted the sample to an experienced editor at Proofed to be proofread and edited with the aim to remove any language mistakes and improve its clarity, concision, and coherence.
The same sample was put through Hemingway Editor to highlight words, sentences, and paragraphs that needed attention.
The results were assessed based on whether the writing quality was improved and writing mistakes were corrected.
After copying and pasting the writing sample into the Hemingway Editor, it immediately received a readability grade of 13, which is 4 grades above the recommended grade of 9.
As shown in the image, here’s the readability grade breakdown:
The first mistake Hemingway Editor made is an incorrect suggestion for a “phrase with a simpler alternative,” (highlighted in purple). In this case, the highlighted word is part of a title being cited in the text, which Hemingway Editor was unable to discern.
Although Hemingway Editor did provide suggestions for alternative words, which you can see in the image below, it was not appropriate for the context.
Other mistakes and errors it didn’t highlight include:
After making changes based on Hemingway Editor’s suggested number of adverbs and use of the passive voice (with no changes made to complex words or phrases because two of the highlights were incorrect suggestions) the grade remained the same: 13.
After making further changes, focusing on sentences that were identified as hard or very hard to read, Hemingway Editor still gave the writing sample a readability grade of 13. And this time, the individual targets changed.
After the first revision, Hemingway Editor indicated that a total of five adverbs was okay but that changed to four or fewer, and the target of seven uses of passive voice was changed to six or fewer.
Although one sentence previously highlighted as “very hard to read” was changed to “hard to read,” another previously unhighlighted sentence was highlighted as “hard to read.”
There was no explanation on the site for these changes.
Overall, Hemingway Editor did a good job of highlighting problem sentences that some readers might have difficulty following. However, it didn’t highlight many grammar, spelling, and punctuation mistakes in the text that are typical of someone who speaks English as an additional language. Furthermore, it didn’t provide any suggestions for changes to the problem sentences.
Essentially, Hemingway Editor didn’t improve the writing sample in any way but merely highlighted sentences and areas that could be difficult to read for some people.
The same writing sample was submitted to an editor at Proofed for editing and proofreading services. After about a day, the writing sample was returned with a Clean copy and a Track Changes copy, showing what changes were made along with comments on areas that were unclear and needed further attention, or explaining why an edit was made.
The editor at Proofed not only eliminated the mistakes that Hemingway Editor missed, but they also improved the overall quality and readability of the text by fixing long, complex sentences and reorganizing and breaking up long paragraphs.
They also left comments on areas that needed further attention from the writer, such as word choice ambiguity, and reduced the writing sample word count from 602 to 536.
Let’s take a closer look:
In this section, the editor at Proofed was able to edit and rewrite sentences as needed to improve the clarity and flow of the paragraph. While Hemingway Editor highlighted the same sentences in this section as “hard to read” or “very hard to read,” the editor at Proofed actually made the necessary changes.
The editor at Proofed corrected all mistakes in the writing sample. They focused on clarity, cohesion, and concision in the text to make sure the main ideas and points were clear and easy to understand. Additionally, the editor ensured that each paragraph flowed in a smooth manner and made changes to accomplish that goal.
Based upon this test, AI writing assistants such as Hemingway Editor, which promise to help you proofread and edit your writing at home, fail to follow through on quality and accuracy in their edits.
The biggest issue we found with Hemingway Editor is that it did not provide suggestions for all the problem areas it highlighted. Additionally, the readability grade lacked consistency, changing grades and requirements without providing any reasons.
The editor at Proofed was able to make edits to the writing sample accurately and consistently, leaving comments with feedback or suggestions for further action from the writer. The editor not only removed writing mistakes, such as grammar, punctuation, and spelling issues, but improved the flow, coherence, and clarity of the writing sample by adding a bulleted list, rewriting problem sentences, and reorganizing paragraphs.
Tired of AI writing assistants failing to provide consistent and reliable editing and proofreading for your business? Schedule a call with us today to learn more about how our experienced, qualified editors can help.
Using AI to Generate Content?
Let’s talk about the support you need.