Rust implementation of surya, a multilingual document OCR toolkit. The implementation is based on a modified version of Segformer, OpenCV, and donut transformer.
Please refer to the original project for more details on licensing of the weights.
This project is still in development, feel free to star and check back.
- image input pre-processing
- detection - segformer
- detection - weights loading
- detection - heatmap and affinity map
- detection - bboxes
- detection - image splitting and stitching
- recognition - swin encoder
- recognition - MoE MBart
- recognition - donut transformer loading
- benchmark
- quantifications
Setup rust toolchain if you haven't yet:
# visit https://rustup.rs/ for more detailed information
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shInstall llvm and opencv (example on Mac):
brew install llvm opencvBuild and install the binary:
# run this first on Mac if you have a M1 chip
export DYLD_FALLBACK_LIBRARY_PATH="$(xcode-select --print-path)/usr/lib/"
# run this first on other Mac
export DYLD_FALLBACK_LIBRARY_PATH="$(xcode-select --print-path)/Toolchains/XcodeDefault.xctoolchain/"
# optionally you can include features like accelerate, metal, mkl, etc.
cargo install --path . --features=cliThe binary when built does not include the weights file itself, and will instead download via the HuggingFace Hub API. Once downloaded, the weights file will be cached in the HuggingFace cache directory.
Check -h for help:
Surya is a multilingual document OCR toolkit, original implementation in Python and PyTorch
Usage: surya [OPTIONS] <IMAGE>
Arguments:
<IMAGE> path to image
Options:
--detection-batch-size <DETECTION_BATCH_SIZE>
detection batch size, if not supplied defaults to 2 on CPU and 16 on GPU
--detection-model-repo <DETECTION_MODEL_REPO>
detection model's hugging face repo [default: vikp/surya_det]
--weights-file-name <WEIGHTS_FILE_NAME>
detection model's weights file name [default: model.safetensors]
--config-file-name <CONFIG_FILE_NAME>
detection model's config file name [default: config.json]
--non-max-suppression-threshold <NON_MAX_SUPPRESSION_THRESHOLD>
a value between 0.0 and 1.0 to filter low density part of heatmap [default: 0.35]
--extract-text-threshold <EXTRACT_TEXT_THRESHOLD>
a value between 0.0 and 1.0 to filter out bbox with low heatmap density [default: 0.6]
--bbox-area-threshold <BBOX_AREA_THRESHOLD>
a pixel threshold to filter out small area bbox [default: 10]
--recognition-batch-size <RECOGNITION_BATCH_SIZE>
recognition batch size, if not supplied defaults to 8 on CPU and 256 on GPU
--recognition-model-repo <RECOGNITION_MODEL_REPO>
recognition model's hugging face repo [default: vikp/surya_rec]
--output-dir <OUTPUT_DIR>
output directory, under which the input image will be generating a subdirectory [default: ./surya_output]
--polygons
whether to output polygons json file
--image
whether to generate bbox image
--heatmap
whether to generate heatmap
--affinity-map
whether to generate affinity map
--device <DEVICE_TYPE>
device type, if not specified will try to use GPU or Metal [possible values: cpu, gpu, metal]
--verbose
whether to enable verbose mode
-h, --help
Print help
-V, --version
Print version
You can also use this to control logging level:
export SURYA_LOG=warn # or debug, warn, etc.This lib is also published as a trait for other rust projects to use.