Datalab Marker v2 vs. MinerU, Docling, and Liteparse: Benchmark Breakdown

Datalab releases Marker 2an entire rewrite of the open supply doc transformation pipeline. Marker converts PDF, picture, PPTX, DOCX, XLSX, HTML, and EPUB information to markdown, JSON, HTML, or chunks. The Datalab staff rebuilt it round three parts that shipped final month. Surya OCR 2, Quick format mannequin with 20M parametersand rebuilt pdf textual content That is 3x quicker than the earlier one.

The principle outcomes are: olmOCR benchAllen AI’s third-party benchmark. marker 2 Balanced mode scored 76.0% general and 83.5% for Born Digital PDF. Maintains 2.9 pages/sec on a single B200 GPU. That is greater than 5 instances the throughput of MinerU’s pipeline backend, which scores 72.7% at 0.54 pages per second. On the identical harness, the docking rating at 2.1 pages/second is 50.3%.

What’s new in Marker 2?

marker 2 Expose three conversion paths as an alternative of 1.

• Balanced — Surya VLM handles the format and re-OCRs the complete web page every time embedded textual content is invalid. Highest quality on GPU. 76.0% olmOCR bench.
• Quick — light-weight rf-detr/onnx format detector and pdftext with minimal use of surgical VLM. 66.6%and less expensive.
• –disable_ocr — Pure textual content layer extraction. There are not any VLM calls in any respect. Runs totally on the CPU. 43.6%23.7 pages/sec.

The present mode is Gadget appropriate by default: Balanced on GPU, quick on CPU/MPS, overridable with -mode. Full CPU help is the second structural change. quick –disable_ocr doesn’t require a GPU or an inference server and reads columns, tables, and headers on the CPU even on 20M format fashions.

The third change is an architectural change and is what drives the throughput numbers. Many skinny CPU employees share a single Surya inference server. The guardian course of units the funds for VLM concurrency throughout processes, so throughput is scaled based on server capability slightly than per-process VRAM. Datalab reviews that balanced mode maintains ~2.9 pg/s for a single stream price of ~0.3 pg/s on the identical {hardware}.

Breaking modifications are value flagging earlier than upgrading. Python 3.10 or later is now required. Packaging from Poetry ultraviolet gentlemakes use of hatchling as a construct backend, however pip set up marker-pdf stays unchanged. Structured extract converters and extractors have been eliminated. Datalab factors customers to hosted APIs or –use_llm workflows as an alternative.

comparability

The scoring standards is olmOCR bench From Ai2: 1,403 PDFs with roughly 8,400 go/fail unit assessments overlaying formulation rendering, desk construction, studying order, headers and footers, and outdated scans. The general rating is a macro common throughout eight classes, calculated by the official olmOCR bench checker. Throughput is maintained simultaneous pg/s on one B200 host as an alternative of single stream delay.

A observe about provenance. Though olmOCR-bench is a third-party benchmark for Ai2, all scores and throughput numbers under are primarily based on Datalab’s personal runs. All of that is reproducible by way of the open harness of the Marker repository. This repository ships with Marker’s personal runners, in addition to competing runners from MinerU, Docling, and LiteParse.

These numbers additionally replicate a mix of paperwork from one benchmark measured on a single {hardware} configuration, so outcomes from your personal doc might fluctuate. Groups evaluating these methods might want to run the harness in opposition to their very own corpora. That is the one approach to know the way the 4 groups rank when it comes to the paperwork they really course of.

Marker 2 vs MinerU

MinerU’s pipeline backend is the closest architecturally. Each selectively learn PDF textual content layers and OCR. In general scores, Marker Steadiness led 76.0 to 72.7. For born-digital paperwork, the 2 are successfully tied: 83.5 vs. 83.3.

Separation is throughput. Marker Steadiness sustains 2.9 pg/s versus MinerU’s 0.54 pg/s, a 5.4x distinction in larger scores. Marker quick maintains 7.4 pg/s, which is about 13.7 instances the pipeline price of MinerU, however its rating is 6.1 factors under MinerU.

MinerU has VLM backend,Datalab reveals a better rating than the pipeline backend. Its backend is a full-page VLM method and isn’t included on this desk. AI groups evaluating MinerU might want to benchmark its paths independently.

Marker 2 vs Docling

Docgling is the widest margin within the GPU pipeline. Marker stability is 76.0 to 50.3 general, with a lead of 83.5 to 64.0 for born digital, and it additionally operates quicker at 2.9 pg/s vs. 2.1 pg/s. In keeping with Datalab, Docling runs in a default pipeline that makes use of textual content layers for born-digital pages and OCR for picture areas.

Docling’s counterpoint just isn’t precision, however breadth of governance and codecs. The code base is MIT licensedborn at IBM Analysis and hosted as a undertaking. LF AI and information infrastructure. Its enter listing extends previous paperwork to audio and e-mail codecs as nicely.

Marker 2 vs. LiteParse

LiteParse from the LlamaIndex staff is a Rust doc parser. They don’t compete on the identical axis. It scored 22.4 general on CPU and 20.4 off OCR. In the meantime, Marker’s CPU-only rating was 43.6.

Nevertheless, LiteParse reviews with OCR disabled 1721 pages/sec — Roughly 73 × Marker CPU mode. It is a trade-off. Marker’s quick –disable_ocr runs a 20M format mannequin on the CPU and nonetheless recovers the construction. This is the reason we greater than double the rating for plain textual content dumps. LiteParse has no format mannequin and collapses on non-linear issues.

Comparability of Marker 2 and full-page VLM layers

The Datalab staff emphasizes that Marker is designed as a pipeline, not a VLM, making it clear that they’re separate instruments. For this analysis, the hosted chandra 2 The rating is 85.8, whereas the Gemini Flash 3.5 rating by way of API is 76.4. Datalab’s Chandra repository additionally has Ai2’s olmOCR 2 at 82.4 and dots.ocr 1.5 at 83.9 in separate tables. With regards to scanning, computationally intensive pages, and reaching the best accuracy, the VLM layer outperforms all listed pipelines.

Marker’s balanced mode reduces general efficiency to Gemini Flash 3.5 by simply 0.4 factors and outperforms Gemini Flash 3.5 by a distinction of 83.5 to 79.1 on born digital paperwork with out requiring per-page API calls.

Conduct by class

Relying on the mode you select, not solely your rating but additionally your failure profile will change. Every row is one olmOCR bench class, scored throughout all three modes. Arithmetic has sharp edges. Quick mode reads formulation from the PDF textual content layer as an alternative of VLM-OCR processing, so arXiv math drops from 83.9 to 23.4 and –disable_ocr scores 0.0 by design. Aside from the 2 math classes, the outdated scan has the weakest break up of any mode, with a most rating of 43.2.

license

That is the place the 4 methods differ most for business groups.

• marker: The code is Apache 2.0. Mannequin weighting makes use of the modified AI Pubs OpenRAIL-M license. Free for analysis, private use, and startups with lower than $5 million in funding/income. Moreover, business use of the weights requires a paid license.
• Minor U: At present beneath the MinerU open supply license with extra phrases primarily based on Apache 2.0. If it exceeds 100 million MAU or $20 million in month-to-month income, a separate business license is required, and on-line companies constructed on prime of it should disclose that reality.
• Dokling: MIT, mannequin licenses are tracked individually within the unique package deal.
• LiteParse: run-llama open supply. LlamaParse is positioned as a paid cloud go for laborious paperwork.

Use Case – Comparability

You do not select a instrument primarily based solely on the rating. Decided by corpus sort, {hardware}, license band, and output format. Strive the interactive picker under to filter 10 deployment eventualities by constraints and instruments to see which parser suits your use case.

Necessary factors

• Marker 2’s balanced rating is 76.0% at 2.9 pg/s on the olmOCR bench, which is greater than 5 instances the pipeline throughput of MinerU.
• Outperforms docking in each axes directly: 76.0% vs. 50.3%, 2.9 pg/s vs. 2.1 pg/s.
• LiteParse trades pace for construction. It is 1721 pg/s with OCR off, however with CPU it is 20.4% in comparison with Marker’s 43.6%.
• Quick mode with –disable_ocr runs totally on the CPU at 23.7 pg/s with out an inference server.
• Licensing is dividing the sector. Docling is MIT, MinerU is free for month-to-month income as much as $20 million, and Marker requires a paid license for weights above $5 million.
• All benchmarks and throughput numbers are shipped with reproducible benchmarks/harnesses.

Interactive dynamic clarification