Reading handwriting on forms requires a two-step approach:
1. Finding the text
Finding the text uses advanced image processing techniques. Doctors don’t tend to write between the lines, and they don’t use the same writer’s style. Therefore, a set of algorithms is used to detect text boundaries and to homogenize the text’s look and feel.
The image-processing algorithms are also required to extract a good training set for the AI models. Examples of such algorithms are:
- Fetching the boundary box of a region of interest (ROI)
- Detecting the number of text lines within the ROI
- The splitting of text into words
- Creating uniform pen thickness of the text
- Suppression of background noise
Reduction of colours
- Matching against templates
2. Reading the text
Once found on the image, the reading of the text is performed by custom-trained AI, based on neural networks and optimized for handwriting. The architecture of the neural network combines recurrent and convolutional layers.
The custom-trained AI performs much better than available third-party generalized models, because they are trained for the specific business case and know the business jargon very well.
Both the image processing, which is CPU-intensive, and the transcribing of the handwriting, which is RAM-intensive, are offered as configurable, stateless micro-services accessible through a RESTfull API. They are integrated with a front-end and back-end application, and orchestration through an external workflow engine.
The production system is run completely in the local data centre. The AI model trainings are performed using Vertex AI on the Google Cloud Platform.