# documents

id: 9dce673a-a5e2-416a-a848-e03c807ab8a0
filename: 2503.11576v1.pdf
content_type: application/pdf
created_at: 2026-03-20 20:38:16.673268+00:00
storage_path:
/Users/pjmalandrino/Documents/Pro/workspace/poc/Docling-Studio/document-parser/uploads/75483ae0-2151-44da-b354-9bc91d6953d5_2503.11576v1.pdf


# analysis_jobs

id: 705de2e3-4dfc-4948-b16f-d5a973212c0c
document_id: 9dce673a-a5e2-416a-a848-e03c807ab8a0
status: COMPLETED
started_at: 2026-03-20 20:38:28.015250+00:00
completed_at: 2026-03-20 20:38:42.677803+00:00
created_at: 2026-03-20 20:38:28.007628+00:00
content_markdown:
## SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

Ahmed Nassar 1 , Andres Marafioti 2 , Matteo Omenetti 1 , Maksym Lysak 1 , Nikolaos Livathinos 1 , Christoph Auer 1 , Lucas Morin 1 , Rafael Teixeira de Lima 1 , Yusik Kim 1 , A. Said Gurbuz 1 , Michele Dolfi 1 , Miquel Farré 2 , Peter W. J. Staar 1

IBM Research 1 , HuggingFace 2

https://huggingface.co/ds4sd/SmolDocling-256M-preview Version 1.0, March 14 th 2025

## Abstract

Weintroduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms - significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon.

## 1 Introduction

For decades, converting complex digital documents into a structured, machine-processable format has been a significant technical challenge. This challenge primarily stems from the substantial variability in document layouts and styles, as well as the inherently opaque nature of the widely used PDF format, which is optimized for printing rather than semantic parsing. Intricate layout styles and visually challenging elements such as forms, tables, and complex charts can significantly impact the reading order and general understanding of documents. These problems have driven extensive research and development across multiple domains of computer science. On one hand, sophisticated ensemble systems emerged, which decompose the conversion problem into several sub-tasks (e.g. OCR, layout analysis, table structure recognition, classification) and tackle each sub-task independently. Although such systems can achieve high-quality results for many document types while maintaining relatively low computational demands, they are often difficult to tune and generalize.

On the other hand, in the recent past, great interest has developed around large foundational multimodal models that can solve the whole conversion task in one shot, while simultaneously offering flexible querying and parametrization through prompts. This approach has been made possible by the advent of multimodal pre-training of large visionlanguage models (LVLMs), which opened up a vast array of opportunities for leveraging diverse data sources, including PDF documents. However, the literature on this topic highlights a significant gap in the availability of high-quality and open-access datasets suitable for training robust multi-modal models for the task of document understanding. Furthermore, relying on LVLMs may introduce common issues associated with such models, including hallucinations and the use of significant computational resources, making them impractical from both a quality and cost perspective.

Figure 1: SmolDocling/SmolVLM architecture. SmolDocling converts images of document pages to DocTags sequences. First, input images are encoded using a vision encoder and reshaped via projection and pooling. Then, the projected embeddings are concatenated with the text embeddings of the user prompt, possibly with interleaving. Finally, the sequence is used by an LLM to autoregressively predict the DocTags sequence.

<!-- image -->

In this work, we outline how we close the gaps left by publicly available datasets and establish a training approach to achieve end-to-end, full-featured document conversion through a vision-language model, which we name SmolDocling . Our main contributions can be summarized as follows:

- SmolDocling: An ultra-compact VLM for end-to-end document conversion, based on Hugging Face's most recent SmolVLM-256M [56], which is between 5 and 10 times smaller in parameters than comparable VLMs tuned on document understanding tasks. Hence, we reduce the computational complexity to roughly the same order of magnitude as typical representatives of ensemble methods. Uniquely, SmolDocling learns and produces a unified document representation that captures content, structure and spatial location of all elements in a page.
- We augment existing document pre-training datasets with additional feature annotations, and create new datasets for tasks with insufficient coverage in open multi-modal training datasets, including code listings, equations, and chart understanding. Importantly, we construct full-page ground truth data that combines key annotation features such as layout, table structure, code listings, charts, and formulas in their natural context, and derive instruction datasets from all training datasets used.
- We introduce DocTags , a document markup format optimized to efficiently represent the full content and layout characteristics of a document, inspired by [55].
- We evaluate the achieved prediction quality of SmolDocling on several tasks, including comparisons to popular community models.

## 2 Related Work

## 2.1 Large Vision-Language Models (LVLMs)

With the advent of large language models (LLMs) [74, 66, 82, 32, 90, 4, 8, 25, 2] characterized by their powerful contextual understanding and adaptability, several approaches have emerged to extend them into large visionlanguage models. Some proprietary, closed-source LLMs have been adapted into multimodal large language models (MLLMs), such as GPT-4o [65], Gemini [80], and Claude 3.5 [5], demonstrating exceptional capabilities across various modalities, including vision. Open-source methods are actively advancing to match the capabilities of proprietary MLLMs. BLIP-2 [47] is one of the earliest works combining a vision encoder with a frozen LLM (OPT [102] or FlanT5 [14]) using a lightweight transformer (Q-former [101]). Building upon BLIP-2, MiniGPT-4 [105] integrated a frozen visual encoder with a frozen LLM (Vicuna [85]) using a Q-Former network and a single projection layer. LLaVA [52, 51] employs a similar architecture but uses a minimal adapter layer. LLaVA-OneVision [45] and LLaVA-NeXTInterleave [46] build upon LLaVA to support multi-image, higher resolution, and video understanding. For handling high resolutions and text-image compositions and comprehension, InternLM-XComposer introduces a family of models [99, 100, 20, 19] designed specifically for these tasks. Qwen-VL [8] introduces a position-aware adapter to address efficiency issues caused by the vision encoder generating long image feature sequences. Qwen2.5-VL [9]

employs windowed attention with 2D rotary positional embeddings to process native-resolution inputs efficiently and integrates an vision-language merger for dynamic feature compression.

In this work, we build upon the SmolVLM architecture approach [56]. Precisely, we adopt SmolVLM-256M, a compact variant of base SmolVLM-2.2B model, inspired by the architecture of Idefics3 [41] which employs the shape-optimized SigLIP encoder as its visual backbone. It applies an aggressive pixel shuffle strategy to compress visual features, reduces the number of image hidden states and introduces new special tokens to separate sub-images and improve tokenization efficiency.

## 2.2 State of the Art in Document Understanding

Several powerful document understanding solutions have emerged on the market in the past decade as commercial cloud offerings on hyperscalers [24, 62, 3, 6], frontier models such as GPT-4o [65] or Claude [5], or as open-source libraries such as Docling [53], Grobid [1], Marker [67], MinerU [86] or Unstructured [84]. Typical tasks that document understanding encompasses are document classification [96, 29], OCR [48], layout analysis [95, 70], table recognition [103, 55, 78], key-value extraction [29, 35, 31], chart understanding [49, 57, 50, 26, 60, 93, 94, 13, 58, 33, 34], equation recognition [77, 68, 18, 97, 23], and more.

Ensemble systems, such as the aforementioned open-source libraries [53, 1, 67, 86], implement a pipeline in source code, which conditionally applies specialized, single-task models on a given input document, composing the prediction results into a meaningful document representation. Typically, each task entails human-crafted handling of preprocessing and post-processing logic (i.e. setting confidence thresholds for layout detections, matching layout elements to text cells, conditionally applying models).

Conversely, multi-task models aim at providing a single model capable of handling various document-understanding related tasks simultaneously, benefiting from shared context and shared representations learned across these different tasks. OCR-reliant methods, such as LayoutLM [96, 95, 29] and UDOP [79], use text extracted from an external OCR engine, along with image and text bounding box locations, as input. In contrast, OCR-free methods such as Donut [35], Dessurt [17], DocParser [76], Pix2Struct [43] are transformer-based models trained end-to-end to take an image as input and directly output text. Large Vision Language Models (LVLMs) such as LLaVA [51], LLaVAOneVision [45], UReader [98], Kosmos-2 [69], and Qwen-VL [8] all leverage various vision encoders, projection adapters, and LLMs. While they may use different image patching techniques and instruction-tuning datasets to handle a range of vision tasks, they commonly evaluate their document understanding capabilities on datasets like DocVQA [59] and mPLUG-DocOwl 1.5 datasets [27], primarily focusing on question answering and reasoning.

Our work focuses primarily on document understanding tasks related to conversion and structural recognition, aiming to represent documents with high accuracy and completeness. The closest works to our approach are Nougat [12], DocOwl 2 [28], GOT [89], and Qwen2.5-VL [9]. DocOwl 2 and GOT are dedicated to precise conversion and recognition of document structures. On one hand, DocOwl 2 employs a dynamic shape-adaptive cropping module to process high-resolution images, feeding them into a ViT vision encoder. A dedicated module compresses the processed visual features before feeding them into a LLaMa-based LLM. On the other hand, GOT focuses on converting diverse elements-such as text, formulas, molecular diagrams, tables, charts, sheet music, and geometric shapes-into structured formats. It achieves a lightweight architecture by utilizing a SAM vision encoder [37], a linear projection layer, and a Qwen-0.5B LLM. Building on these approaches, our work not only extends the range of tasks but also broadens the types of documents handled, moving beyond a strictly scientific focus to support a more diverse array of document formats. In parallel, Qwen2.5-VL introduces the Omni-Parsing strategy, that integrates multiple document elements into a unified HTML-based representation. This format encodes layout bounding box information and reading order for various document elements such as paragraphs and charts.

## 3 SmolDocling

Building on Hugging Face's SmolVLM [56], SmolDocling converts document page images into sequences representing their content and structure. It produces a unified page representation (DocTags markup) that is optimized for the LLM backbone and interpretable by Docling [53]. This new DocTags format covers the reproduction of captions, charts, forms, code, equations, tables, footnotes, lists, page footers and headers, section headings, and text. SmolDocling both recognizes the type and location of these elements as well as performs OCR within them. Our approach enables full-page conversion with reading-order and hierarchical linking between elements (e.g., linking captions with figures and tables), providing bounding boxes for each component, supporting isolated predictions on individual elements (e.g., cropped tables) and grouping for lists and nested lists.

Fig. 3. OTSL description of table structure: A - table example; B - graphical repre- sentation of table structure; C - mapping structure on a grid; D - OTSL structure

ucccc c c

c c

2 - horizonta merces. C.

I - vertical merges: "C", U

· c

Table 2. TSR and cell detection results compared between OTSL and HTML on the PubTabNet (22], FinTabNet [21] and PubTables-IM [14] data sets using Table-

Former [9) (with enc=6, dec=6, heads=8).

Data set

Language

PubTabNet

FinTabNet

Pub'Tables-1M

OTSL

HTML

OTSL

HTML

OTSL

HTMLI

5.3

15.3 Qualitative Results

The OTS: vocabulary is comprised of the following tokens.

"C" cell - a new table cell that either has or does not have cell content

"L" cell - left-looking cell, merging with the left neighbor cell to create a

"U" cell - up-looking cell, merging with the upper neighbor cell to create a span

"X" cell - cross cell, to merge with both left and upper neighbor cells

"NL" - new-line, switch to the next row.

A notable attribute of OTSL is that it has the capability of achieving lossless

D

expression using a data frame called dif in R where you want to summarize x by month:....

require(plyr)

df &lt;- data. frame(|

x = runif(120, 1, 168),

2 = runif(120, 1.7, 20.7), y = runif(120, 7, 334).

month = rep(c(5,6,7,8),30), week = sample(1:4, 120, TRUE)

ddply(df, · (month, week), sumarize, sd = round(sd(x), 2))

‹picture&gt;&lt;loc\_102&gt;&lt;loc\_85&gt;&lt;loc\_416&gt;&lt;loc\_160&gt;~

&lt;flow\_chart&gt;~

‹/picture&gt;~

&lt;otsl&gt;&lt;loc\_63&gt;&lt;loc\_78&gt;&lt;loc\_436&gt;&lt;loc\_158&gt;*|

‹ched›Data set. &lt;ched&gt;Language&lt;ched&gt;TEDs. &lt;Icel&gt;.

<!-- image -->

mean = round(mean(x), 2),

Figure 2: DocTags format describes key features of document elements: type of an element (text, picture, table, code, etc.), location on the page, and content. Nested tags convey additional information: pictures and tables may nest captions, table structure is represented in OTSL tags, lists nest list items, code and pictures carry a classification. All DocTags output is taken from SmolDocling predictions, artificial line-breaks and dots (...) were inserted for better readability. A, B, C - snippets of various pages from [55], D - from [92]

## 3.1 Model Architecture

The model architecture is illustrated in Figure 1. SmolVLM-256M relies on SigLIP base patch-16/512 (93M) as visual backbone and compared to the 2.2B version of the same model, its training data was rebalanced to emphasize document understanding (41%) and image captioning (14%), combining The Cauldron [42], Docmatix [41] datasets with the addition of MathWriting [23]. It uses lightweight variant of the SmolLM-2 family (135M) [2] as the language backbone and employs a radical pixel shuffle method that compresses each 512x512 image patch into 64 visual tokens. Last but not least, tokenization efficiency was also improved in SmolVLM-256M by increasing the pixel-to-token ratio to 4096 pixels per token and introducing special tokens for sub-image separators.

## 3.2 DocTags

To train VLMs for standardized document conversion, we introduce the DocTags markup format, illustrated in Figure 3. Inspired by OTSL [55], DocTags define a structured vocabulary of unambiguous tags and rules that explicitly separate textual content from document structure. This disentanglement improves performances of Image-to-Sequence models by minimizing confusions. In contrast, direct conversion to standard formats such as HTML or Markdown is often lossy and ambiguous, lacking explicit structural markup, failing to preserve page layout context, and increasing total tokens count. Pieces of text content that represent basic elements in DocTags are enclosed in XML-style tags. Tags define document block types such as: text, caption, footnote, formula, title, list\_item, page\_footer, page\_header, picture, section\_header, otsl, code, document\_index, etc. Each element can nest additional location tags encoding its position on the page as a bounding box, represented in DocTags as &lt;loc\_ x1 &gt;&lt;loc\_ y1 &gt;&lt;loc\_ x2 &gt;&lt;loc\_ y2 &gt; .

Special blocks, such as tables and images, nest additional descriptors for captions, tabular structure, or image classes. To encode table structures, we fully include the OTSL vocabulary into DocTags . OTSL has the advantage of being minimalist and supporting cell spans and header information. Its tokens are interleaved with text, allowing

,&lt;ched&gt;mAP(0.75)&lt;ched&gt;Inference..&lt;nl&gt;~

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion simultaneous encoding of both table structure and content. In order to promote robust visual-semantic alignment in our document understanding pipeline, we maintain a uniform DocTags representation for cropped page elements (e.g., tables, code, equations) that is identical to their full-page counterparts. This consistency ensures that the model can leverage similar visual cues whether an element is presented in isolation or in the context of an entire page, thereby strengthening the learning signal for both computer vision and downstream machine learning tasks.

For a complete list of tags and detailed description of the format, consult the appendix.

## 3.3 Model Training

We employ a curriculum learning approach to progressively align our model for document conversion, ensuring faster convergence. As an initial step, we incorporate DocTags as tokens in our tokenizer. To align the LLM part we freeze the vision encoder and train only the remaining network to adapt it to the new output format which it hasn't seen before. To ensure comprehensive coverage of all DocTags, we maintain a balanced mix of tasks and data types during training. Next, we unfreeze the vision encoder and train the model on our pretraining datasets (see Sec. 4.1), along with all task-specific conversion datasets, including tables, code, equations, and charts. Finally, we fine-tune using all available datasets.

## 4 Data

For document understanding tasks, each page image represents the visual layout of textual elements-such as paragraphs, formulas, tables, code, or figures with embedded textual content. Publicly available datasets that unify all these characteristics remain limited, and existing corpora often suffer from small sample sizes or fragmented formats. In this section, we explain how we mitigate these issues in both pre-training and task-specific training scenarios.

## 4.1 Pre-training datasets

In the domain of document understanding, currently available open-access document pre-training datasets include OCR-IDL [10], IIT-CDIP [44], CCpdf [83], and WordScape [88]. OCR-IDL and IIT-CDIP, industry-sourced from the 1990s, provide only page images and OCR text. CCpdf and WordScape, derived from Common Crawl, focus on raw PDFs and Word documents, respectively. While WordScape enriches documents with layout, table structure, and topic information via XML markup, its visual variability is limited. Notably, none of these datasets offer rich annotations for all page elements.

DocLayNet-PT. Given the lack of public multimodal document pre-training data with sufficient annotation features, we constructed a new document pre-training dataset named DocLayNet-PT. DocLayNet-PT is a 1.4M pages dataset extracted from the DocFM dataset [25] of unique PDF documents, sourced from CommonCrawl, Wikipedia, and business-related documents. We focused on this dataset to look for documents including visually distinct content such as equations, tables, code and charts with colorful layouts. This dataset was augmented with weak annotations through a series of processing steps. First, we applied PDF parsing to all pages to obtain the text layout including positional information using Docling [7, 53]. Next, we performed an enrichment step to obtain an enhanced, machine-friendly representation of each page. Specifically, we provide annotations for layout elements, table structure, language, topic, and figure classification on each page in DocTag format.

Docmatix. To preserve SmolVLM's original DocVQA capabilities, we adapted the same weak annotation strategy used for DocLayNet-PT and applied it to the 1.3M documents inside Docmatix dataset [41]. In addition to the 9.5 million DocVQA questions, we introduced another instruction requiring the full conversion of multi-page documents to DocTags.

## 4.2 Task-specific Datasets

Layout. To optimize prediction quality for document layout and table structure, we sampled 76K pages from DocLayNet-PT. Each page underwent human annotation and rigorous quality review, resulting in a high-quality ground-truth dataset. Of these, 60K pages were utilized for fine-tuning, constituting what we refer to as DocLayNet v2. Additionally, we extracted 63K pages from WordScape, specifically filtering for pages containing both text and tables, and utilized their inherently structure-preserving format as a reliable source of ground truth. To further enhance the model's adaptability to diverse layouts, colors, and fonts, we synthetically generated 250K pages using content from Wikipedia, creating the dataset known as SynthDocNet. This synthetic dataset enables the controlled generation of various document types with precise annotations. All datasets share a unified annotation format and class definitions, ensuring consistency and comprehensive document representation.

Tables. For the table structure recognition task we trained our model on public datasets such as PubTables-1M [78], FinTabNet [103], WikiTableSet [54] and tabular information extracted from WordScape documents. We transformed table structure information from the original ground-truth to OTSL format [55]. Then, we interleaved each cell tag with ground-truth text. This yields a condensed sequence which represents the structure of the table (including merged cells, and header declaration) along with its textual content.

Charts. For chart reconstruction, several datasets are publicly available [58, 33, 34, 93] that consist of both synthetic and web-scraped charts. However, these datasets are limited by either the quantity of real-world data or the visual diversity of synthetic examples. This has prompted recent studies [50, 26, 60, 94, 93, 13, 57] to crawl or render additional datasets, which unfortunately are rarely shared publicly. To address these gaps, we generated our own chart dataset comprising four types of charts: line, pie, bar, and stacked bar. Using data from 90,000 tables sourced from the FinTabNet dataset [103], we generated a comprehensive dataset containing 5,000 line charts, 380,000 pie charts, 380,000 bar charts, and 77,000 stacked bar charts. Each data point was rendered using three different visualization libraries (details provided in the Appendix), yielding a total of 2.5 million visually diverse charts.

Code. Technical books and scientific documents frequently include code snippets, making it essential for documentprocessing models to accurately interpret code structure, particularly indentation, which carries semantic significance in certain programming languages. Several publicly available datasets provide extensive collections of code snippets with permissive licenses in text format [38, 71], typically aggregated from sources like GitHub. However, a common limitation is that these datasets contain only text-based code snippets, lacking any image-based representation. To address this gap, we leveraged LaTeX and Pygments [16] to generate visually diverse renderings of code (details provided in the Appendix). Our dataset includes 9.3M million rendered code snippets generated at 120 dpi, covering 56 programming languages.

Equations. Accurately parsing mathematical formulas is crucial for document-processing models, as formulas often contain nuanced structures and semantics critical for scientific understanding. To effectively train our model for formula transcription, we combined publicly available datasets with a custom-rendered dataset. The public datasets [18, 22, 11, 23] collectively comprise approximately 730k unique formulas. In addition, we extracted 4.7 million formulas from arXiv by applying a regular expression to the source LaTeX code. The extracted formulas underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized LaTex code. Subsequently, these normalized equations were visually rendered using LaTeX (details provided in the Appendix). Overall, our dataset includes 5.5 million unique formulas, each rendered at a resolution of 120 dpi.

Table 1: Datasets. Overview of the datasets used for the training and evaluation of SmolDocling. Datasets we contribute as part of this publication are denoted with (*).

| Dataset Name                     | Size   | Type       |
|----------------------------------|--------|------------|
| TheCauldron [42]                 | 2.63M  |            |
| DocLayNet-PT-Instruct            | 1.62M  | Multi Task |
| DocMatix w/ Doc Conversion* [41] | 952K   |            |
| SynthChartNet*                   | 2.5M   |            |
| PlotQA [61]                      | 225K   |            |
| FigureQA [33]                    | 100K   | Chart      |
| Unichart [57]                    | 600K   |            |
| ChartQA [58]                     | 28K    |            |
| DocLayNet-PT                     | 1.40M  |            |
| Doc4MLlaVa - RVLCDIP             | 742K   |            |
| SynthDocNet                      | 375K   | Conversion |
| DocLayNet [70]                   | 81K    |            |
| DocLayNet v2                     | 76K    |            |
| WordScape-PT                     | 63K    |            |
| PubTable1M [78]                  | 1M     |            |
| WordScape Tables [88]            | 207K   | Table      |
| WikiTableSet (EN) [54]           | 204K   |            |
| FinTabNet [103]                  | 90K    |            |
| SynthFormulaNet*                 | 5.5M   | Formula    |
| SynthCodeNet*                    | 9.3M   | Code       |

## 4.3 Document Instruction Tuning datasets

To reinforce the recognition of different page elements and introduce document-related features and no-code pipelines, we leveraged rule-based techniques and the Granite-3.1-2b-instruct [25] LLM. Using samples from DocLayNet-PT pages, we generated one instruction by randomly sampling layout elements from a page. These instructions included tasks such as 'Perform OCR at bbox,' 'Identify page element type at bbox,' and 'Extract all section headers from the page.' In addition, we train with Cauldron [42] to avoid catastrophic forgetting with the amount of conversation datasets introduced.

| Method             | Size   |   Edit Distance ↓ |   F1-score ↑ |   Precision ↑ |   Recall ↑ |   BLEU ↑ |   METEOR ↑ |
|--------------------|--------|-------------------|--------------|---------------|------------|----------|------------|
| Full-page          |        |                   |              |               |            |          |            |
| Qwen2.5 VL [9]     | 7B     |              0.56 |         0.72 |          0.8  |       0.7  |     0.46 |       0.57 |
| GOT [89]           | 580M   |              0.61 |         0.69 |          0.71 |       0.73 |     0.48 |       0.59 |
| Nougat (base) [12] | 350M   |              0.62 |         0.66 |          0.72 |       0.67 |     0.44 |       0.54 |
| SmolDocling (Ours) | 256M   |              0.48 |         0.8  |          0.89 |       0.79 |     0.58 |       0.67 |
| Code listings      |        |                   |              |               |            |          |            |
| SmolDocling (Ours) | 256M   |              0.11 |         0.92 |          0.94 |       0.91 |     0.87 |       0.89 |
| Equations          |        |                   |              |               |            |          |            |
| Qwen2.5 VL [9]     | 7B     |              0.22 |         0.89 |          0.91 |       0.87 |     0.68 |       0.77 |
| GOT [89]           | 580M   |              0.11 |         0.95 |          0.95 |       0.96 |     0.85 |       0.91 |
| Nougat (base) [12] | 350M   |              0.62 |         0.6  |          0.6  |       0.53 |     0.33 |       0.41 |
| SmolDocling (Ours) | 256M   |              0.11 |         0.95 |          0.96 |       0.95 |     0.83 |       0.89 |

Table 2: Structured document text recognition. We evaluate OCR performance and document formatting on DocLayNet, focusing on textual elements, excluding tables. Text accuracy is measured through multiple metrics (Edit Distance, F1-Score, Precision, Recall, BLEU and METEOR) as proposed in [12].

## 5 Experiments

In this section, we assess the performance of SmolDocling on document understanding tasks, and compare the results to previously published models and baselines. Capturing the qualities and defects of SmolDocling and other models in comparable numbers is a non-trivial effort for reasons we outline further below. Therefore, we report results on standard evaluation tasks and metrics independently (see section 5.2), but also provide a qualitative assessment of output characteristics (see section 5.3).

## 5.1 Implementation Details

Our training was conducted using 64 NVIDIA A100 80GB GPUs, with each epoch requiring 38 hours. The model was trained for a total of 4 epochs using the AdamW optimizer with a learning rate of 2 × 10 -4 and 2 × 10 -6 for the vision encoder once unfrozen. We also employed gradient clipping to 1.0 for stability and a warmup ratio of 0.03. For inference, using VLLM [39] on an A100 GPU, SmolDocling achieves a page conversion time of 0.35 seconds while occupying only 0.489 GB of VRAM. The model supports a maximum sequence length of 8,192 tokens and is trained to process up to three pages at a time.

## 5.2 Quantitative Results

To enable valid cross-model comparisons with SmolDocling, we addressed multiple dataset and harmonization challenges:

- Public evaluation datasets lack comprehensive multi-task annotations with diverse layouts. We augmented DocLayNet [70] with text content and reading-order to enable evaluation of both layout analysis and text recognition. Table structure, code listing, formula and chart recognition are evaluated on specific datasets.
- Different models each follow their own conventions for output markup (e.g., Markdown, HTML, DocTags) and produce annotations which do not fully align in semantics. For example, SmolDocling is trained to produce 14 distinct layout-class tags via DocTags , while Qwen2.5-VL combines standard HTML tags and a few explicit class attributes with only partially compatible definitions. GOT and Nougat output Markdown, which can express paragraphs, lists and tables without location annotation.

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

- Input resolution of page images can affect model performance, but is rarely specified in literature. We standardized at 144 DPI (dots per inch), which reproduces enough detail to be perfectly legible by humans while keeping resource demand for inference reasonable.

Text Recognition (OCR). We first evaluate text recognition accuracy of SmolDocling, compared to Nougat, GOT and Qwen2.5-VL (Table 2), to verify accurate character transcription and reading order. The text-similarity metrics, adopted from [12] require plain formatting without non-textual elements. For different tasks, we conduct evaluations on diverse content types: full-page documents (from DocLayNet test set) in Markdown format, code snippets (from our SynthCodeNet dataset) in plaintext, and equations (from Im2Latex-230k [21]) in LaTeX format. For Qwen2.5-VL and SmolDocling, we convert HTML or DocTags output to Markdown while preserving formatting where possible.

Tables are omitted from the full-page output since they do not compare well through Markdown, due to loss of row and column spans. Code snippets must be reproduced with intact line breaks and indentation for accurate scoring. Formulas are evaluated separately in LaTeX format, which is natively produced by all the models tested in this work. The LaTeX outputs from all models, including the ground truth, undergo the same normalization procedure used for generating the SynthFormulaNet dataset. This ensures consistency and fairness in model comparisons.

SmolDocling significantly outperforms GOT, Nougat and also Qwen2.5-VL, which is 27 times larger, on every metric in full-page transcription (see Table 2. For the code parsing task, we report results exclusively for SmolDocling, as we introduce this task for the first time, and the other models were not explicitly trained for it. SmolDocling also demonstrates superior performance compared to Nougat and Qwen2.5-VL in the formula recognition task, closely matching GOT's results.

Layout. We evaluate element localization accuracy using the DocLayNet test set (Table 3), measuring the ability to reconstruct document representations that support content grounding and visual feature extraction. SmolDocling is compared against Qwen2.5-VL-7b [9], currently the only other VLM addressing this task. For a fair comparison, we map each model's label outputs and the ground-truth to six compatible classes: Text, Section Heading, List Item, Table, Picture, and Formula. Overall, SmolDocling outperforms Qwen2.5-VL-7b by a significant margin, however both models score far below a human baseline. This can be partially explained through inherent difficulty of the dataset, as indicated by relatively low human agreement scores. The scores are primarily impacted by moderate recall of element bounding boxes and different degrees of label confusion in both SmolDocling and Qwen2.5-VL. A majority of samples show accurate element bounding boxes as seen in the Appendix.

|                |   SmolDocling |   Qwen2.5-VL |   Human |
|----------------|---------------|--------------|---------|
| Formula        |         0.267 |        0.059 |    0.84 |
| Table          |         0.266 |        0.262 |    0.79 |
| List Item      |         0.296 |        0.09  |    0.87 |
| Section Header |         0.289 |        0.129 |    0.83 |
| Text           |         0.204 |        0.18  |    0.85 |
| Picture        |         0.066 |        0.078 |    0.7  |
| Overall        |         0.231 |        0.133 |    0.82 |

Table 3: Layout analysis comparison. Evaluation of layout analysis task on DocLayNet with a subset of 6 class labels in mAP[0.5:0.95] metric. Additional reference points are provided for an estimate of human performance [70].

Table Structure Recognition. We evaluate the accuracy of recovering table structures, i.e. shaping columns and rows with the correct text content and spans, through the TEDS metric [104], once including table text content and once on structure only with ommitted text (see Table 4). For isolated assessment, we provide cropped table images from FinTabNet [103] and PubTables-1M [78] test sets, which have poor image quality (72 dpi, compression artifacts). SmolDocling performs competitively against significantly larger models, despite challenges with text transcription from the low-resolution image crops, a limitation likely attributable to insufficient training at such resolutions. Scores for structure only TEDS evaluation are therefore much stronger.

Chart Extraction. In this task, we evaluate the ability to convert cropped sections of charts into datapoint tables. All table ouptut is transformed into HTML and compared against the ground-truth HTML using TEDS score, as reported in Table 5. We attribute SmolDocling's performance on charts to the variability in ground-truth styles and the differing interpretations of representations across datasets, which we observe leads to inconsistencies in the model's predictions. Despite its significantly smaller size, SmolDocling achieves a competitive TEDs score.

Overall, SmolDocling achieves leading performance across a wide range of document conversion tasks and outperforms significantly larger models in text recognition, layout analysis and structure extraction. Furthermore, the model's

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

| Method              | Model Size   | FinTabNet   | PubTables-1M   |
|---------------------|--------------|-------------|----------------|
| SmolVLM [56]        | 2.2B         | 0.18        | -              |
| Granite Vision [81] | 2.98B        | 0.54        | -              |
| TableFormer† [55]   | 52M          | 0.89        | 0.84           |
| SmolDocling (Ours)  | 256M         | 0.52 (0.81) | 0.65 (0.88)    |

Table 4: Table structure and cell content reconstruction. Numbers are given in TEDS metric including text content, with structure-only score in brackets where available. We produced results for SmolDocling and TableFormer, other results are reflected here as published. †TableFormer extracts the content through external OCR.

| Model               | Size   | TEDs ↑   |
|---------------------|--------|----------|
| Phi-3.5-vision      | 4B     | 0.40     |
| Granite Vision [81] | 3B     | 0 . 95   |
| SmolVLM [56]        | 2.2B   | 0.02     |
| Molmo-E             | 1B     | 0.54     |
| SmolDocling (Ours)  | 256M   | 0.75     |

Table 5: Chart analysis. We compare SmolDocling to other small-sized VLMs on the chart-to-table task using the TEDS metric. The results in this table are based on the work reported in [25].

high performance in downstream tasks, such as code listing and formula recognition, demonstrates its versatility and real-world applicability. Notably, we establish benchmark results for the code listing recognition, setting a standard for future evaluations.

## 5.3 Qualitative Results

Several characteristics of SmolDocling output are not evident from task-based metrics alone. In this section, we highlight different output characteristics observed in practice. More illustrated samples can be found in the Appendix.

As presented in Figure 3, SmolDocling outputs a sequence of DocTags to encode content, structure and layout. A few notable traits beyond straight content reproduction include: 1) Labeling of page headers and footers, which allows ignoring these repetitive elements in downstream applications. 2) Linking of information, such as captions to tables or pictures, or list elements to parent list through nesting of tags. 3) Preserved line breaks and indentation for code snippets, while line wraps in paragraphs are correctly removed.

Among typical defects found in the output of SmolDocling are missing tags (e.g. absent location tags), malformed structure (such as missing closing tags), and endless repetition of tokens from some point in a page. Both of these phenomena are commonly seen in auto-regressive models and may prevent parsing or rendering the output correctly (see Appendix). Despite these limitations, SmolDocling demonstrates unique robustness advantages compared to ensemble systems like Docling, since conversion output is inferred in a single pass. In the latter, errors may accumulate throughout the ensemble model pipeline. For example, a mislocated table detection will lead to missing or distorted output of table structure and content, since this depends on correct table localization. In contrast, SmolDocling can reproduce the tables correctly, even with incorrectly determined location.

## 6 Conclusion

In this work, we introduce SmolDocling, an efficient and compact VLM optimized for document conversion while providing rich output representation. We also present a suite of new datasets with a unified format for document conversion, including the novel task of code listing transcription. We identify page element localization as a critical area requiring further refinement, where targeted techniques will significantly enhance performance in future iterations. Our results conclusively demonstrate that smaller models with unified, optimized output formats such as DocTags can effectively compete with significantly larger models, establishing a clear pathway for resource-efficient multi-task document understanding models.

## References

- [1] Grobid. https://github.com/kermitt2/grobid , 2008-2025.
- [2] Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf. SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model, 2025.
- [3] Amazon Web Services. Amazon Textract. https://aws.amazon.com/textract/ , 2024. Accessed: 2024-11-11.
- [4] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403 , 2023.
- [5] Anthropic. Claude-3.5. https://www.anthropic.com/news/claude-3-5-sonnet , 2024. Accessed: 202402-11.
- [6] Christoph Auer, Michele Dolfi, André Carvalho, Cesar Berrospi Ramis, and Peter WJ Staar. Delivering Document Conversion as a Cloud Service with High Throughput and Responsiveness. In 2022 IEEE 15th International Conference on Cloud Computing (CLOUD) , pages 363-373. IEEE, 2022.
- [7] Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Nikolaos Livathinos, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Fabian Lindlbauer, Kasper Dinkla, et al. Docling Technical Report. arXiv preprint arXiv:2408.09869 , 2024.
- [8] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 , 2023.
- [9] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025.
- [10] Ali Furkan Biten, Rubèn Tito, Lluis Gomez, Ernest Valveny, and Dimosthenis Karatzas. Ocr-idl: Ocr annotations for industry document library dataset. In European Conference on Computer Vision , pages 241-252. Springer, 2022.
- [11] Lukas Blecher. LaTeX-OCR, 2024. Accessed: 2024-11-09.
- [12] Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint arXiv:2308.13418 , 2023.
- [13] Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. OneChart: Purify the Chart Structural Extraction via One Auxiliary Token. arXiv preprint arXiv:2404.09987 , 2024.
- [14] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research , 25(70):1-53, 2024.
- [15] Pyecharts Contributors. pyecharts: Python library for Echarts, 2024. Accessed: 2024-11-06.
- [16] Pygments Contributors. Pygments: Python syntax highlighter, 2024. Accessed: 2024-11-06.
- [17] Brian Davis, Bryan Morse, Brian Price, Chris Tensmeyer, Curtis Wigington, and Vlad Morariu. End-to-end document recognition and understanding with dessurt. In European Conference on Computer Vision , pages 280-296. Springer, 2022.
- [18] Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M Rush. Image-to-markup generation with coarse-to-fine attention. In International Conference on Machine Learning , pages 980-989. PMLR, 2017.
- [19] Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model. arXiv preprint arXiv:2401.16420 , 2024.
- [20] Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Songyang Zhang, Haodong Duan, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Zhe Chen, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Kai Chen, Conghui He, Xingcheng Zhang, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD. arXiv preprint arXiv:2404.06512 , 2024.
- [21] Gregory Eritsyan. Im2LaTeX 230K. https://www.kaggle.com/datasets/gregoryeritsyan/ im2latex-230k , 2023. Accessed: 2025-03-06.
- [22] Hugging Face. latex-formulas Dataset, 2024. Accessed: 2024-11-09.
- [23] Philippe Gervais, Asya Fadeeva, and Andrii Maksai. Mathwriting: A dataset for handwritten mathematical expression recognition. URL https://arxiv. org/abs/2404.10690 , 2024.
- [24] Google Cloud. Document AI. https://cloud.google.com/document-ai , 2024. Accessed: 2024-11-11.
- [25] IBM Granite Team. Granite 3.0 Language Models, 2024.

- [26] Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483 , 2023.
- [27] Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403.12895 , 2024.
- [28] Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding. arXiv preprint arXiv:2409.03420 , 2024.
- [29] Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia , pages 4083-4091, 2022.
- [30] J. D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science &amp; Engineering , 9(3):90-95, 2007.
- [31] Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) , volume 2, pages 1-6. IEEE, 2019.
- [32] Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 , 2024.
- [33] Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300 , 2017.
- [34] Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. arXiv preprint arXiv:2203.06486 , 2022.
- [35] Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Donut: Document understanding transformer without ocr. arXiv preprint arXiv:2111.15664 , 7(15):2, 2021.
- [36] Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton. PubChem 2019 update: improved access to chemical data. Nucleic Acids Research , 47(D1):D1102-D1109, 10 2018.
- [37] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015-4026, 2023.
- [38] Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. The Stack: 3 TB of permissively licensed source code. Preprint , 2022.
- [39] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023.
- [40] Greg Landrum. RDKit: Open-Source Cheminformatics Software. http://www.rdkit.org/. Accessed: 1 January 2023.
- [41] Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and better understanding vision-language models: insights and future directions., 2024.
- [42] Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models?, 2024.
- [43] Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Conference on Machine Learning , pages 18893-18912. PMLR, 2023.
- [44] David Lewis, Gady Agam, Shlomo Argamon, Ophir Frieder, David Grossman, and Jefferson Heard. Building a test collection for complex document information processing. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval , pages 665-666, 2006.
- [45] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 , 2024.
- [46] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895 , 2024.
- [47] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.
- [48] Han Lin, Peng Yang, and Fanlong Zhang. Review of scene text detection and recognition. Archives of computational methods in engineering , 27(2):433-454, April 2020.
- [49] Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. arXiv preprint arXiv:2212.10505 , 2022.

- [50] Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. arXiv preprint arXiv:2212.09662 , 2022.
- [51] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved Baselines with Visual Instruction Tuning, 2023.
- [52] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. In NeurIPS , 2023.
- [53] Nikolaos Livathinos, Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Kasper Dinkla, Yusik Kim, Shubham Gupta, Rafael Teixeira de Lima, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, and Peter W. J. Staar. Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion, 2025.
- [54] Nam Ly., Atsuhiro Takasu., Phuc Nguyen., and Hideaki Takeda. Rethinking Image-Based Table Recognition Using Weakly Supervised Methods. pages 872-880, 2023.
- [55] Maksym Lysak, Ahmed Nassar, Nikolaos Livathinos, Christoph Auer, and Peter Staar. Optimized Table Tokenization for Table Structure Recognition. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 37-50, Cham, 2023. Springer Nature Switzerland.
- [56] Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf. SmolVLM: Redefining small and efficient multimodal models. 2025.
- [57] Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arXiv:2305.14761 , 2023.
- [58] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244 , 2022.
- [59] Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 2200-2209, 2021.
- [60] Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. Chartassisstant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. arXiv preprint arXiv:2401.02384 , 2024.
- [61] Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. PlotQA: Reasoning over Scientific Plots. In The IEEE Winter Conference on Applications of Computer Vision (WACV) , March 2020.
- [62] Microsoft Azure. AI Document Intelligence. https://azure.microsoft.com/en-us/products/ ai-services/ai-document-intelligence , 2024. Accessed: 2024-11-11.
- [63] Lucas Morin, Martin Danelljan, Maria Isabel Agea, Ahmed Nassar, Valery Weber, Ingmar Meijer, Peter Staar, and Fisher Yu. MolGrapher: Graph-based Visual Recognition of Chemical Structures. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 19552-19561, October 2023.
- [64] Lucas Morin, Valéry Weber, Gerhard Ingmar Meijer, Fisher Yu, and Peter W. J. Staar. PatCID: an open-access dataset of chemical structures in patent documents. Nature Communications , 15(1):6532, Aug 2024.
- [65] OpenAI. Hello GPT-4O. https://openai.com/index/hello-gpt-4o/ , 2024. Accessed: 2024-02-09, 2024-0211, 2024-02-12.
- [66] R OpenAI. Gpt-4 technical report. arxiv 2303.08774. View in Article , 2(5), 2023.
- [67] Vik Paruchuri. Marker: Convert PDF to Markdown Quickly with High Accuracy. https://github.com/VikParuchuri/marker, 2024.
- [68] Vik Paruchuri. Texify. https://github.com/VikParuchuri/texify , 2024. Accessed: 2024-11-11.
- [69] Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824 , 2023.
- [70] Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S. Nassar, and Peter Staar. DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 3743-3751, New York, NY, USA, 2022. Association for Computing Machinery.
- [71] Ruchir Puri, David Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, Shyam Ramji, Ulrich Finkler, Susan Malaika, and Frederick Reiss. CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks, 2021.
- [72] Edward O. Pyzer-Knapp, Matteo Manica, Peter Staar, Lucas Morin, Patrick Ruch, Teodoro Laino, John R. Smith, and Alessandro Curioni. Foundation models for materials discovery - current state and future directions. npj Computational Materials , 11(1):61, Mar 2025.
- [73] Yujie Qian, Jiang Guo, Zhengkai Tu, Zhening Li, Connor W. Coley, and Regina Barzilay. MolScribe: Robust Molecular Structure Recognition with Image-to-Graph Generation. Journal of Chemical Information and Modeling , 63(7):1925-1934, Apr 2023.

- [74] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research , 21(140):1-67, 2020.
- [75] Kohulan Rajan, Henning Otto Brinkhaus, M. Isabel Agea, Achim Zielesny, and Christoph Steinbeck. DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications. Nature Communications , 14(1):5045, Aug 2023.
- [76] Johannes Rausch, Octavio Martinez, Fabian Bissig, Ce Zhang, and Stefan Feuerriegel. Docparser: Hierarchical document structure parsing from renderings. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 4328-4338, 2021.
- [77] Felix M Schmitt-Koopmann, Elaine M Huang, Hans-Peter Hutter, Thilo Stadelmann, and Alireza Darvishy. MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition. IEEE Access , 2024.
- [78] Brandon Smock, Rohith Pesala, and Robin Abraham. PubTables-1M: Towards comprehensive table extraction from unstructured documents. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4624-4632, 2022.
- [79] Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mohit Bansal. Unifying vision, text, and layout for universal document processing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19254-19264, 2023.
- [80] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.
- [81] Granite Vision Team, Leonid Karlinsky, Assaf Arbelle, Abraham Daniels, Ahmed Nassar, Amit Alfassi, Bo Wu, Eli Schwartz, Dhiraj Joshi, Jovana Kondic, Nimrod Shabtay, Pengyuan Li, Roei Herzig, Shafiq Abedin, Shaked Perek, Sivan Harary, Udi Barzelay, Adi Raz Goldfarb, Aude Oliva, Ben Wieles, Bishwaranjan Bhattacharjee, Brandon Huang, Christoph Auer, Dan Gutfreund, David Beymer, David Wood, Hilde Kuehne, Jacob Hansen, Joseph Shtok, Ken Wong, Luis Angel Bathen, Mayank Mishra, Maksym Lysak, Michele Dolfi, Mikhail Yurochkin, Nikolaos Livathinos, Nimrod Harel, Ophir Azulai, Oshri Naparstek, Rafael Teixeira de Lima, Rameswar Panda, Sivan Doveh, Shubham Gupta, Subhro Das, Syed Zawad, Yusik Kim, Zexue He, Alexander Brooks, Gabe Goodhart, Anita Govindjee, Derek Leist, Ibrahim Ibrahim, Aya Soffer, David Cox, Kate Soule, Luis Lastras, Nirmit Desai, Shila Ofek-koifman, Sriram Raghavan, Tanveer Syeda-Mahmood, Peter Staar, Tal Drory, and Rogerio Feris. Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence, 2025.
- [82] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.
- [83] Michał Turski, Tomasz Stanisławek, Karol Kaczmarek, Paweł Dyda, and Filip Graliński. CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 348-365, Cham, 2023. Springer Nature Switzerland.
- [84] Unstructured.io Team. Unstructured.io: Open-Source Pre-Processing Tools for Unstructured Data. https://unstructured.io, 2024. Accessed: 2024-11-19.
- [85] Vicuna. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. https://vicuna.lmsys. org/ , 2023.
- [86] Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. MinerU: An Open-Source Solution for Precise Document Content Extraction, 2024.
- [87] Michael L Waskom. Seaborn: statistical data visualization. Journal of Open Source Software , 6(60):3021, 2021.
- [88] Maurice Weber, Carlo Siebenschuh, Rory Butler, Anton Alexandrov, Valdemar Thanner, Georgios Tsolakis, Haris Jabbar, Ian Foster, Bo Li, Rick Stevens, et al. WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data. Advances in Neural Information Processing Systems , 36, 2024.
- [89] Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model. arXiv preprint arXiv:2409.01704 , 2024.
- [90] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824-24837, 2022.
- [91] David Weininger. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences , 28(1):31-36, Feb 1988.
- [92] Wes McKinney and the Pandas Development Team. Pandas: powerful Python data analysis toolkit, Release 1.4.4. https: //pandas.pydata.org/pandas-docs/version/1.4/pandas.pdf , 2022.
- [93] Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Junchi Yan, and Yu Qiao. StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding. arXiv preprint arXiv:2309.11268 , 2023.

- [94] Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, et al. Chartx &amp; chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185 , 2024.
- [95] Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, et al. Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 , 2020.
- [96] Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery &amp; data mining , pages 1192-1200, 2020.
- [97] Zuoyu Yan, Xiaode Zhang, Liangcai Gao, Ke Yuan, and Zhi Tang. ConvMath: a convolutional sequence network for mathematical expression recognition. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 4566-4572. IEEE, 2021.
- [98] Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. arXiv preprint arXiv:2310.05126 , 2023.
- [99] Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition. arXiv preprint arXiv:2309.15112 , 2023.
- [100] Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLMXComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output. arXiv preprint arXiv:2407.03320 , 2024.
- [101] Qiming Zhang, Jing Zhang, Yufei Xu, and Dacheng Tao. Vision Transformer with Quadrangle Attention. arXiv preprint arXiv:2303.15105 , 2023.
- [102] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. URL https://arxiv. org/abs/2205.01068 , 3:19-0, 2023.
- [103] Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 697-706, 2021.
- [104] Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes. Image-based table recognition: data, model, and evaluation. arXiv preprint arXiv:1911.10683 , 2019.
- [105] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 , 2023.

Multi Task

PubTablelM

SynthFormulaNet

Figure 3: Training datasets. A treemap visualization of dataset sizes contributing to the training of SmolDocling, categorized by type. Each rectangle represents a dataset, with its area proportional to the total dataset size. Each color indicates a dataset type. 'Multi-Task" refers to datasets that include multiple different tasks.

<!-- image -->

## A Additional Datasets Details

## A.1 SynthChartNet

SynthChartNet includes four distinct types of charts: line, pie, bar, and stacked bar, generated using three visualization libraries: Matplotlib [30], Seaborn [87], and Pyecharts [15]. Each data set was rendered through all three libraries, producing a final dataset of 2.5 million visually diverse charts. The rendering process was designed to leverage the full range of features available in these libraries, ensuring visual variety while maintaining the plausibility of the data presented. Figure 4 depicts 10 samples from SynthChartNet.

## A.2 SynthCodeNet

To render the code snippets of SynthCodeNet, we employed two rendering libraries: LaTeX and Pygments [16]. LaTex, in conjunction with its listings package, enabled the generation of code renderings similar to those commonly seen in academic and technical publications, while Pygments facilitated renderings with aesthetics resembling those of IDEs. The rendering process utilized the complete set of features provided by these libraries to enhance visual diversity while preserving the plausibility of the presented data. This included randomizing various stylistic parameters such as font, font size, style, weight, line spacing, tab size, background color, line numbering (including randomizing the starting line number), and programming language-specific syntax highlighting. Figure 5 shows 4 samples from SynthCodeNet.

## A.3 SynthFormulaNet

The extracted formulas of SynthFormulaNet underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized code. The normalization process consisted of the following major steps:

1. Token Filtering and Removal : Unnecessary or redundant LaTeX tokens were removed or replaced based on predefined policies. This included:
- Eliminating unwanted tokens.
- Replacing tokens with equivalent ones for consistency.

ChartQA

MolGrapher Synthetic 300k WordScape-PocLayNe[nichart

PlotQA

SynthDocNet

FinTabNet

600

500 -

400-

300 -

200

100-

1700

borond suel

1800

1900

Property plant and equipment -

Iccumulated depre..

Cross asset volur

Construction in progress in millions except per share

Total revenues

Income (loss) from continuing

Income from continuing

Net income (loss)

Total assets

Short-term debt

Total debt (including capital

Total stockholders' equity

2000

Enteray Texal

-** nteray New or eani

Enteray Hississippl

Interay Art.

2100

2200

-Ind

2300

12.31

band improvements

Duldings and building

- machinery and equipmen

13.85

16.92

Total Cash Dividends Paid

Toral Cash Widends Paid der Common Share

2011

2012

2013

2014

Total unrecognized

2500 5000 7500 10000 12500 15000 17500 20000

<!-- image -->

1Year

United States loYear

· 5Year|

<!-- image -->

O..................... . ... . . ... ..... ........ .... ·

2075

2100

2125

2150

2175

2200

2225

<!-- image -->

First econd

Third

<!-- image -->

7.69

Figure 4: Samples from SynthChartNet.

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

24.7%

2250

25.3%

· 3Year

24.4%

25.7%

16..

20.I

#pragma once

#include &lt;sdm/world/solvable\_by\_dp. hpp&gt;

#include &lt;sdm/utils/value\_function/initializer.hpp&gt;

namespace sdm namespace

{

class

{

registry protected:

typedef std:: map&lt;std:: string, std:: shared

\_ptr&lt;

world, Config config) &gt; map\_type;

Initializer &gt; (*) (std:: shared\_ptr&lt;SolvableByDP&gt;

static map\_type container;

initializer sane: Stage the app

A20RE, NEBAT NAVE: Y0218431-T5E-APP...

................

1t: contains (gathub. event.pull\_request labela.*.name, 'stage*)

steps:

- uses: actions/checkout (vl

- name: pm install and build webpack run:

npm instal.

- uses: actions/upload-artifactênain apm run build

with:

path: public/

name: webpack artifacts

Busld-Docker-Image:

runs-on: ubuntu-latest

```
public: static std:: vector<std:: string > available (): static std:: shared_ptr<Initializer > make(std :: string name, std:: shared_ptr<SolvableByDP> world, Config config = {}); } :1 template <class TInitializer > std:: shared_ptr<Initializer > createlnstance (std:: shared_ptr<SolvableByDP> world, Config config) return std:: make_shared<TInitializer >(world, config): package br.com.biot. integra caopagarmeapi.modulos.jwt.utids import br.com.biot. integracaopagarmeapi.config.exception.ValidacaoExceptipn import io.jsonwebtoken.Claims import lombok.extern.slf4j.Slf4j public class JwtCampoUtil{ public static String getCampold(Claims claims) {| tryi return (String) claims.get("id"); loge to tro do tentar recuperar I do JWT: 0'; claims toString): throw new ValidacaoException("Erro ao tentar recuperar o campo ID do Token."): public static String getCampo(String campo, Claims claims) { return (String) claims.get(campo); sagetro euro de entar recuperar o campo f] do IT: 0', campo, claims toString): throw new ValidacaoException( String.format("Erro ao tentar recuperar o campo %s do Token.", campo) 29| needs: build name: Build image and store in GitHub Packages steps: - name: Checkout uses: actions/checkout iva - name: Download built artifact uses: actions/download-arts factinain path: public - name: create inage and store in Packages uses: nattdavsa0351/actions/docker-gpr01.3.0 with: repo-token: 3{ (secrets.GITHUB,TOKEN) ) runneone wountuel tent needs: Build-Docker-Image name: Deploy app container to Azure steps: - name: "Login via Azure CLI" MIRA creda: 91{ SeCretA.AZURE_CREDENTIALS 1) - uses: azure/docker-logan@va with: logan-server: #{(env. IMAGE_REGISTRY_URL)) def main (filename, days) : · array I of buckets 0-8, each representing with that counter value fish_by_counter = [O for i in range (9)] for counter in read_counters (filename): fish_by_counter [counter) += 1 the number of fish for i in range (days): fish_by_counter · spawn (fish_by_counter) print (f"After {days) days: {sum(fish_by_counter)} fish") def spawn (fish_by_counter): # rotate left res = fish_by_counter (1:) + fish_by_counter [:1] # fish that were in bucket O are now new fish in bucket 8. restore the original fish into bucket 6. res [6] += res [8] return res def read_counters (filename) : with open (filename) as £ : return [int (counter) split (',')] for counter in f. readline ().strip(). -_name.. == '__main_.': main('input.txt', 80) main ('input.txt', 256)
```

Figure 5: Samples from SynthCodeNet.

kejnKm kejnKm

kejnKm

=I+II + II.

<!-- formula-not-decoded -->

d ≤ 1

Y

4.30

4.31

4.32

<!-- formula-not-decoded -->

(x)

<!-- formula-not-decoded -->

Figure 6: Samples from SynthFormulaNet.

SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

2. Structural Normalization : The equation structure was standardized by:
- Normalizing spacing between tokens.
- Ensuring consistent use of tokens that require one or two braces.
- Standardizing left and right delimiters.
3. Token Simplification : Redundant tokens and patterns were simplified, including:
- Collapsing consecutive redundant tokens (e.g., repeated primes, dots, or excessive spacing).
- Converting special constructs such as \Big into \left and \right for consistency.

The extracted equations were subsequently rendered using LaTex at 120 dpi. Several aspects of the rendering procedure have been randomized to ensure the visual diversity of the dataset. Each formula was rendered using a randomly selected font from a set of 85 possible options, with variations in font size, style, weight, and line spacing. Additionally, with a certain probability, each formula was enclosed within an "align" environment, allowing for the inclusion of a randomly generated equation number on either the left or right side of the equation. The equation number itself was chosen at random from a diverse set of possibilities, including small and large numbers, integers, floating-point values, or letters. Furthermore, it could be enclosed in single or double parentheses, brackets, or curly braces and was sometimes preceded by text labels such as 'Eqn.", 'Eqn", 'Eq.", or 'Equation". Figure 6 depicts 6 samples from SynthFormulaNet.

## A.4 DocTags vocabulary

Here we provide a reference to a complete vocabulary of DocTags for SmolDocling , and their meaning.

We represent tags in an XML-like style of notation. Some tags have an opening and closing part, and wrap around textual content (or other tags) for example: &lt;text&gt;hello world&lt;/text&gt; . Other tags are standalone and do not have a closing tag; they mark a special instruction, for example: &lt;page\_break&gt;

The complete DocTags snippet can represent a page or multiple pages, wrapped into &lt;doctag&gt;...&lt;/doctag&gt; . When content represents multiple pages, we use &lt;page\_break&gt; tag as a separator.

Within the scope of &lt;doctag&gt; we place high-level, tags that wrap around the textual content of certain document blocks and identify the type of the block. These tags are: &lt;text&gt; , &lt;caption&gt; , &lt;footnote&gt; , &lt;formula&gt; , &lt;title&gt; , &lt;page\_footer&gt; , &lt;page\_header&gt; , &lt;picture&gt; , &lt;section\_header&gt; , &lt;document\_index&gt; , &lt;code&gt; , &lt;otsl&gt; , &lt;list\_item&gt; , &lt;ordered\_list&gt; , &lt;unordered\_list&gt;

Each element can nest additional standalone location tags that encode its position on the page as a bounding box, represented in DocTags as &lt;loc\_ x1 &gt;&lt;loc\_ y1 &gt;&lt;loc\_ x2 &gt;&lt;loc\_ y2 &gt; . Every x,y pair of coordinates belongs to a fixed grid. In our case we used integer values in the range from 0 to 500 which proportionally maps to width and height of the page.

&lt;otsl&gt; element contains table representation with following OTSL tags, that are standalone and used to describe structure of a table: &lt;fcel&gt; , &lt;ecel&gt; , &lt;lcel&gt; , &lt;ucel&gt; , &lt;xcel&gt; , &lt;nl&gt; . These tags follow rules as described in [55], we additionally differentiate OTSL notion of a cell into &lt;fcel&gt; - full cell, or cell which contain content, and &lt;ecel&gt; - empty cell. We also augment tabular structure information with extra tags &lt;ched&gt; , &lt;rhed&gt; , &lt;srow&gt; that replace &lt;fcel&gt; to describe cells which belong to column and row headers of the table, as well as table section accordingly. We mark such headers when ground truth about table headers from the underlying dataset is available.

&lt;list\_item&gt; elements are placed within &lt;ordered\_list&gt; or &lt;unordered\_list&gt; and define weather it's enumerated (ordered) list or not.

&lt;picture&gt; and &lt;otsl&gt; elements by themselves can encapsulate &lt;caption&gt; tag if appropriate picture or table has it's own caption. This way we can connect extra descriptive information to illustrations and tables in the document.

&lt;code&gt; elements contain pieces of code as a content, and as such, respect tabulation and line breaks. Code elements also contains special standalone classification tag -&lt;\_ programming-language \_&gt; where programminglanguage is an appropriate programming language name. Supported values (57 in total) for programming language names are: Ada , Awk , Bash , bc , C , C# , C++ , CMake , COBOL , CSS , Ceylon , Clojure , Crystal , Cuda , Cython , D , Dart , dc , Dockerfile , Elixir , Erlang , FORTRAN , Forth , Go , HTML , Haskell , Haxe , Java , JavaScript , Julia , Kotlin , Lisp , Lua , Matlab , MoonScript , Nim , OCaml , ObjectiveC , Octave , PHP , Pascal , Perl , Prolog , Python , Racket , Ruby , Rust , SML , SQL , Scala , Scheme , Swift , TypeScript , unknown , VisualBasic , XML , YAML

Within &lt;picture&gt; elements we include picture classification with extra standalone tags &lt; image\_class&gt; . We classify images from our datasets into following categories: natural\_image , pie\_chart , bar\_chart , line\_chart ,

flow\_chart , scatter\_chart , heatmap , remote\_sensing , chemistry\_molecular\_structure , chemistry\_markush\_structure , icon , logo , signature , stamp , qr\_code , bar\_code , screenshot , map , stratigraphic\_chart , cad\_drawing , electrical\_diagram

## B Additional Details about the SmolDocling

Table 6: Supported Conversion Instructions.

| Instruction                 | Description                                                               |
|-----------------------------|---------------------------------------------------------------------------|
| Full conversion             | Convert this page to docling.                                             |
| Chart                       | Convert chart to table (e.g., <chart> ).                                  |
| Formula                     | Convert formula to LaTeX (e.g., <formula> ).                              |
| Code                        | Convert code to text (e.g., <code> ).                                     |
| Table                       | Convert table to OTSL (e.g., <otsl> ).                                    |
| No-Code Ac- tions/Pipelines | OCR the text in a specific location: <loc_155><loc_233><loc_206><loc_237> |
| No-Code Ac- tions/Pipelines | Identify element at: <loc_247><loc_482><loc_252><loc_486>                 |
| No-Code Ac- tions/Pipelines | Find all 'text' elements on the page, retrieve all section headers.       |
| No-Code Ac- tions/Pipelines | Detect footer elements on the page.                                       |

Input Image

OH

Cl

· 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR

2.5-VL-72B and SmolDocling on a simple molecule image.

GOT-OCR 2.0

NH,

Figure 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR 2.0, Qwen2.5-VL-72B and SmolDocling on a simple molecule image.

<!-- image -->

## C Additional Qualitative Analysis

## C.1 Molecule Image Recognition Experiment

Extracting molecule images from documents has potential to accelerate research in chemistry and materials discovery [72]. Some recent document understanding models, such as GOT-OCR 2.0 [89] and Qwen2.5-VL [9], claim to perform molecule image recognition in documents. Following this direction, we experimented with training SmolDocling for molecule image recognition on full document pages and cropped images. In order to predict molecule structures (graphs) with SmolDocling, we use the molecule string identifier named SMILES [91].

Datasets. For training on cropped images, we use the MolGrapher-Synthetic-300K dataset [63]. The dataset contains 300K synthetic samples. It is created using real chemical-structures retrieved from the database PubChem [36] which are then synthetically drawn using the library RDKit [40]. For training of full pages, we use Doc2Mol-80K, a subset of the PatCID dataset [64]. Our subset contains 80 000 real patent documents published in the United-States between 2022 and 2024.

Evaluation. Figure 7 shows an example of predicted molecules for a simple molecule image. While some models capture high-level molecular features, they consistently fail to reconstruct its detailed structure. Notably, this molecule is trivial for specialized models such as MolGrapher [63], MolScribe [73] or DECIMER [75], which accurately recognize its structure. Reducing the gap between general document understanding models and specialized models may require incorporating a specialized vocabulary for SMILES tokens, explicitly encoding atoms and bonds, and extending the SMILES sequence with localization tokens for each atom.

## C.2 Layout analysis samples

To illustrate and compare the visual grounding capabilities of SmolDocling, a set of sample predictions with SmolDocling and Qwen2.5-VL from DocLayNet [70] is shown in Table 7. Note that the element location results are independent from the correct reproduction of document content and structure.

Ground-Truth

46"

peR, heath and safety

Final gonn so yoR and convertible loan

tor save did exit) in the Opoon promp pur puse the Enter key:

trollowing display appears.

apposoices, tress later, yange Sample form nurter 3 in the Tet "description' prompt.

Hollowing display appears

Tyle doices, tress toler

Create earlay..

Moneyle Actior: Press the Enter key.

310.000,000 pal during the 2015

Sign untain Change in matters suusequent to

the end of the financial year

40

SmolDocling

2wen2.5-VL

Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. Win following, display appears we want yourse

Mum: F4 To 11S

<!-- image -->

Ground-Truth

301 Workers engarod in cential oe emeriency works pass ntial or emergency works" includes-

gunmpletions, maintenance, repairs, and asset replacement activities.

Quality

*/ Supie (England) Regularions 20164), the Water Supply Nu Qual

Low crane Ixcince tacans the looser of a seucrare Ixcice under sochon 178% of tho

Water Industry Act 1991(e):

we crape services" has the meaning givee in sction 2191) of the Water Industry Actl

48/ We industy ice 19 lar the mating given in seties 17ACT) and 2I9C) of the nak managemcat on bchall of

pay one nvironment Agency, or)

tal tood dinory to three.

Cle the purposes of sub-paragraph l wyneed and "coastal crosion" have the meanings given in section I of the Plood and

Water Management Act 20100h);

pg/w9, management" has the meaning gives in section 3 of that Acci.

229,n29 1) Workers engaged in essential or emergency works- we we stricity interccenector,

regree trict heat network as delined in regulation 2 of the Heat Network (Meicnng and

Billing) Regulations 2014(j).

Billing) Regulations 2014.

put mao ommissicaling, maintenance and repair of industrial machinery for use on al

Рд (1 4/3130. There as Do trialt areading envie:

I canations of the saddle point are thie to lowing)

(11))

Mll9. C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on the support of p. We can prove that these equations lead to a singular support for p. We

do this by contradiction.

woRd then that

397199e first two terms obviously are cancelled. Notice that

3'iMat away from 2' - O we have that

91a + 03) - Pª (log(=*) + log(=*)) - 0|

peese the function is harmonic: a sum of i hodonsorplise plus an antiliolomorphic plece:

RI, is is casy to show that (F-) log(a - 6) o d°(a - 6). With these, we would find that p = 0. This argument contradicts that fp = 1, so the assumption that p is smooth does

not work. This argument has boen given before in (5, 23), and we are presenting it bere for completeness

siddhat this indicates is that the demity distribution of narticks le sinoular. Since tot

9 = 0 the original probability distribution is spherically symmetric, it is natural to assume that the associated distribution is spherically summetric. The radius was computed in 118

Pitchas also been chocked macrically in 6. 7. A geocralization to all Calabi- Yau conc conforinal field tboory singularities for this result can be found in (23), where it was seen that

this procedure matches vurious otber supergravity calculations in the abeence of symmetry.

anger q &gt; 0 but very small, we expect that the distribution does not change much, except in the region :' ≥ 0. Thus we expoct that the distribution gives a five dimensional manifold of

R°, possibly with a boundary, and that the region bear 2' = O has no density of eigenvalues present. After all, the effoctive potential repels all particles from the locus :' = 0. This

smolDocling

Y/I"Woekers engaged in essential or emergensy works.

K/""ential or emergency works" inclades

Greedons, maintenance, repairs, and asset replacement activities,

2016(a).

Repain 2 a. he lies Super it Quie)

Supply (Water

Quality

Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the

Qwen2.5-VL

21. (1) Workers encaned in cascatial or emergency works—

"O) "emal or emergency works" includes-

0) mspoctos, mamienance, repam, and asset replacement actioes and analysis of walcr

Watch

Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the

2016(a).

Supply (Wa

Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. "Wille C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on tbe suppost of p. We can prove that these equations lead to a singular support for p. We do this by contradiction.

do this by contradiction.

<!-- image -->

DEVOTION TO SERVICE

Net incorre heinrat let tormente

Total astets

Working capital

Shareholders equity

Common shares outstanding

$1171

$ 875.228

$ 265,375

$ 436,616

15.971

38,104

$ 1.197,188

$ 295,648

$ 451.169

36.520

Giving back to the and where our

employees live, work, and care for their

families is a serious and dedicated

commitment.

FINANCIAL HIGHLIGHTS

VISA's ongoing commitment to investing in core products and focus on managing costs helped generate a strong finish in 2011.

CROCURR TUAL SALES

BY CORE PRODUCT GROUP

© Fund Gin A Fume Detectior

· Poutis Car Outection

@ Other hodets

60152

1.91

S1015092

$ 287,079

$ 433,666

36,493

DEVOTION TO SERVICE

"Giving back to the and where our

-employees live, work, and care for their

families is a serious

Giving back to the and where our

employees live, work, and care for their

families is a serious

Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures.

340.9

DEVOTION TO SERVICE

22%

<!-- image -->

content_html:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8"/>
<title>75483ae0-2151-44da-b354-9bc91d6953d5_2503.11576v1</title>
<meta name="generator" content="Docling HTML Serializer"/>
<style>
    html {
        background-color: #f5f5f5;
        font-family: Arial, sans-serif;
        line-height: 1.6;
    }
    body {
        max-width: 800px;
        margin: 0 auto;
        padding: 2rem;
        background-color: white;
        box-shadow: 0 0 10px rgba(0,0,0,0.1);
    }
    h1, h2, h3, h4, h5, h6 {
        color: #333;
        margin-top: 1.5em;
        margin-bottom: 0.5em;
    }
    h1 {
        font-size: 2em;
        border-bottom: 1px solid #eee;
        padding-bottom: 0.3em;
    }
    table {
        border-collapse: collapse;
        margin: 1em 0;
        width: 100%;
    }
    th, td {
        border: 1px solid #ddd;
        padding: 8px;
        text-align: left;
    }
    th {
        background-color: #f2f2f2;
        font-weight: bold;
    }
    figure {
        margin: 1.5em 0;
        text-align: center;
    }
    figcaption {
        color: #666;
        font-style: italic;
        margin-top: 0.5em;
    }
    img {
        max-width: 100%;
        height: auto;
    }
    pre {
        background-color: #f6f8fa;
        border-radius: 3px;
        padding: 1em;
        overflow: auto;
    }
    code {
        font-family: monospace;
        background-color: #f6f8fa;
        padding: 0.2em 0.4em;
        border-radius: 3px;
    }
    pre code {
        background-color: transparent;
        padding: 0;
    }
    .formula {
        text-align: center;
        padding: 0.5em;
        margin: 1em 0;
        background-color: #f9f9f9;
    }
    .formula-not-decoded {
        text-align: center;
        padding: 0.5em;
        margin: 1em 0;
        background: repeating-linear-gradient(
            45deg,
            #f0f0f0,
            #f0f0f0 10px,
            #f9f9f9 10px,
            #f9f9f9 20px
        );
    }
    .page-break {
        page-break-after: always;
        border-top: 1px dashed #ccc;
        margin: 2em 0;
    }
    .key-value-region {
        background-color: #f9f9f9;
        padding: 1em;
        border-radius: 4px;
        margin: 1em 0;
    }
    .key-value-region dt {
        font-weight: bold;
    }
    .key-value-region dd {
        margin-left: 1em;
        margin-bottom: 0.5em;
    }
    .form-container {
        border: 1px solid #ddd;
        padding: 1em;
        border-radius: 4px;
        margin: 1em 0;
    }
    .form-item {
        margin-bottom: 0.5em;
    }
    .image-classification {
        font-size: 0.9em;
        color: #666;
        margin-top: 0.5em;
    }
</style>
</head>
<body>
<div class='page'>
<h2>SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion</h2>
<p>Ahmed Nassar 1 , Andres Marafioti 2 , Matteo Omenetti 1 , Maksym Lysak 1 , Nikolaos Livathinos 1 , Christoph Auer 1 , Lucas Morin 1 , Rafael Teixeira de Lima 1 , Yusik Kim 1 , A. Said Gurbuz 1 , Michele Dolfi 1 , Miquel Farré 2 , Peter W. J. Staar 1</p>
<p>IBM Research 1 , HuggingFace 2</p>
<p>https://huggingface.co/ds4sd/SmolDocling-256M-preview Version 1.0, March 14 th 2025</p>
<h2>Abstract</h2>
<p>Weintroduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms - significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon.</p>
<h2>1 Introduction</h2>
<p>For decades, converting complex digital documents into a structured, machine-processable format has been a significant technical challenge. This challenge primarily stems from the substantial variability in document layouts and styles, as well as the inherently opaque nature of the widely used PDF format, which is optimized for printing rather than semantic parsing. Intricate layout styles and visually challenging elements such as forms, tables, and complex charts can significantly impact the reading order and general understanding of documents. These problems have driven extensive research and development across multiple domains of computer science. On one hand, sophisticated ensemble systems emerged, which decompose the conversion problem into several sub-tasks (e.g. OCR, layout analysis, table structure recognition, classification) and tackle each sub-task independently. Although such systems can achieve high-quality results for many document types while maintaining relatively low computational demands, they are often difficult to tune and generalize.</p>
<p>On the other hand, in the recent past, great interest has developed around large foundational multimodal models that can solve the whole conversion task in one shot, while simultaneously offering flexible querying and parametrization through prompts. This approach has been made possible by the advent of multimodal pre-training of large visionlanguage models (LVLMs), which opened up a vast array of opportunities for leveraging diverse data sources, including PDF documents. However, the literature on this topic highlights a significant gap in the availability of high-quality and open-access datasets suitable for training robust multi-modal models for the task of document understanding. Furthermore, relying on LVLMs may introduce common issues associated with such models, including hallucinations and the use of significant computational resources, making them impractical from both a quality and cost perspective.</p>
<figure><figcaption><div class="caption">Figure 1: SmolDocling/SmolVLM architecture. SmolDocling converts images of document pages to DocTags sequences. First, input images are encoded using a vision encoder and reshaped via projection and pooling. Then, the projected embeddings are concatenated with the text embeddings of the user prompt, possibly with interleaving. Finally, the sequence is used by an LLM to autoregressively predict the DocTags sequence.</div></figcaption></figure>
<p>In this work, we outline how we close the gaps left by publicly available datasets and establish a training approach to achieve end-to-end, full-featured document conversion through a vision-language model, which we name SmolDocling . Our main contributions can be summarized as follows:</p>
<ul>
<li style="list-style-type: '· ';">SmolDocling: An ultra-compact VLM for end-to-end document conversion, based on Hugging Face's most recent SmolVLM-256M [56], which is between 5 and 10 times smaller in parameters than comparable VLMs tuned on document understanding tasks. Hence, we reduce the computational complexity to roughly the same order of magnitude as typical representatives of ensemble methods. Uniquely, SmolDocling learns and produces a unified document representation that captures content, structure and spatial location of all elements in a page.</li>
<li style="list-style-type: '· ';">We augment existing document pre-training datasets with additional feature annotations, and create new datasets for tasks with insufficient coverage in open multi-modal training datasets, including code listings, equations, and chart understanding. Importantly, we construct full-page ground truth data that combines key annotation features such as layout, table structure, code listings, charts, and formulas in their natural context, and derive instruction datasets from all training datasets used.</li>
<li style="list-style-type: '· ';">We introduce DocTags , a document markup format optimized to efficiently represent the full content and layout characteristics of a document, inspired by [55].</li>
<li style="list-style-type: '· ';">We evaluate the achieved prediction quality of SmolDocling on several tasks, including comparisons to popular community models.</li>
</ul>
<h2>2 Related Work</h2>
<h2>2.1 Large Vision-Language Models (LVLMs)</h2>
<p>With the advent of large language models (LLMs) [74, 66, 82, 32, 90, 4, 8, 25, 2] characterized by their powerful contextual understanding and adaptability, several approaches have emerged to extend them into large visionlanguage models. Some proprietary, closed-source LLMs have been adapted into multimodal large language models (MLLMs), such as GPT-4o [65], Gemini [80], and Claude 3.5 [5], demonstrating exceptional capabilities across various modalities, including vision. Open-source methods are actively advancing to match the capabilities of proprietary MLLMs. BLIP-2 [47] is one of the earliest works combining a vision encoder with a frozen LLM (OPT [102] or FlanT5 [14]) using a lightweight transformer (Q-former [101]). Building upon BLIP-2, MiniGPT-4 [105] integrated a frozen visual encoder with a frozen LLM (Vicuna [85]) using a Q-Former network and a single projection layer. LLaVA [52, 51] employs a similar architecture but uses a minimal adapter layer. LLaVA-OneVision [45] and LLaVA-NeXTInterleave [46] build upon LLaVA to support multi-image, higher resolution, and video understanding. For handling high resolutions and text-image compositions and comprehension, InternLM-XComposer introduces a family of models [99, 100, 20, 19] designed specifically for these tasks. Qwen-VL [8] introduces a position-aware adapter to address efficiency issues caused by the vision encoder generating long image feature sequences. Qwen2.5-VL [9]</p>
<p>employs windowed attention with 2D rotary positional embeddings to process native-resolution inputs efficiently and integrates an vision-language merger for dynamic feature compression.</p>
<p>In this work, we build upon the SmolVLM architecture approach [56]. Precisely, we adopt SmolVLM-256M, a compact variant of base SmolVLM-2.2B model, inspired by the architecture of Idefics3 [41] which employs the shape-optimized SigLIP encoder as its visual backbone. It applies an aggressive pixel shuffle strategy to compress visual features, reduces the number of image hidden states and introduces new special tokens to separate sub-images and improve tokenization efficiency.</p>
<h2>2.2 State of the Art in Document Understanding</h2>
<p>Several powerful document understanding solutions have emerged on the market in the past decade as commercial cloud offerings on hyperscalers [24, 62, 3, 6], frontier models such as GPT-4o [65] or Claude [5], or as open-source libraries such as Docling [53], Grobid [1], Marker [67], MinerU [86] or Unstructured [84]. Typical tasks that document understanding encompasses are document classification [96, 29], OCR [48], layout analysis [95, 70], table recognition [103, 55, 78], key-value extraction [29, 35, 31], chart understanding [49, 57, 50, 26, 60, 93, 94, 13, 58, 33, 34], equation recognition [77, 68, 18, 97, 23], and more.</p>
<p>Ensemble systems, such as the aforementioned open-source libraries [53, 1, 67, 86], implement a pipeline in source code, which conditionally applies specialized, single-task models on a given input document, composing the prediction results into a meaningful document representation. Typically, each task entails human-crafted handling of preprocessing and post-processing logic (i.e. setting confidence thresholds for layout detections, matching layout elements to text cells, conditionally applying models).</p>
<p>Conversely, multi-task models aim at providing a single model capable of handling various document-understanding related tasks simultaneously, benefiting from shared context and shared representations learned across these different tasks. OCR-reliant methods, such as LayoutLM [96, 95, 29] and UDOP [79], use text extracted from an external OCR engine, along with image and text bounding box locations, as input. In contrast, OCR-free methods such as Donut [35], Dessurt [17], DocParser [76], Pix2Struct [43] are transformer-based models trained end-to-end to take an image as input and directly output text. Large Vision Language Models (LVLMs) such as LLaVA [51], LLaVAOneVision [45], UReader [98], Kosmos-2 [69], and Qwen-VL [8] all leverage various vision encoders, projection adapters, and LLMs. While they may use different image patching techniques and instruction-tuning datasets to handle a range of vision tasks, they commonly evaluate their document understanding capabilities on datasets like DocVQA [59] and mPLUG-DocOwl 1.5 datasets [27], primarily focusing on question answering and reasoning.</p>
<p>Our work focuses primarily on document understanding tasks related to conversion and structural recognition, aiming to represent documents with high accuracy and completeness. The closest works to our approach are Nougat [12], DocOwl 2 [28], GOT [89], and Qwen2.5-VL [9]. DocOwl 2 and GOT are dedicated to precise conversion and recognition of document structures. On one hand, DocOwl 2 employs a dynamic shape-adaptive cropping module to process high-resolution images, feeding them into a ViT vision encoder. A dedicated module compresses the processed visual features before feeding them into a LLaMa-based LLM. On the other hand, GOT focuses on converting diverse elements-such as text, formulas, molecular diagrams, tables, charts, sheet music, and geometric shapes-into structured formats. It achieves a lightweight architecture by utilizing a SAM vision encoder [37], a linear projection layer, and a Qwen-0.5B LLM. Building on these approaches, our work not only extends the range of tasks but also broadens the types of documents handled, moving beyond a strictly scientific focus to support a more diverse array of document formats. In parallel, Qwen2.5-VL introduces the Omni-Parsing strategy, that integrates multiple document elements into a unified HTML-based representation. This format encodes layout bounding box information and reading order for various document elements such as paragraphs and charts.</p>
<h2>3 SmolDocling</h2>
<p>Building on Hugging Face's SmolVLM [56], SmolDocling converts document page images into sequences representing their content and structure. It produces a unified page representation (DocTags markup) that is optimized for the LLM backbone and interpretable by Docling [53]. This new DocTags format covers the reproduction of captions, charts, forms, code, equations, tables, footnotes, lists, page footers and headers, section headings, and text. SmolDocling both recognizes the type and location of these elements as well as performs OCR within them. Our approach enables full-page conversion with reading-order and hierarchical linking between elements (e.g., linking captions with figures and tables), providing bounding boxes for each component, supporting isolated predictions on individual elements (e.g., cropped tables) and grouping for lists and nested lists.</p>
<p>Fig. 3. OTSL description of table structure: A - table example; B - graphical repre- sentation of table structure; C - mapping structure on a grid; D - OTSL structure</p>
<p>ucccc c c</p>
<p>c c</p>
<p>2 - horizonta merces. C.</p>
<p>I - vertical merges: "C", U</p>
<p>· c</p>
<p>Table 2. TSR and cell detection results compared between OTSL and HTML on the PubTabNet (22], FinTabNet [21] and PubTables-IM [14] data sets using Table-</p>
<p>Former [9) (with enc=6, dec=6, heads=8).</p>
<p>Data set</p>
<p>Language</p>
<p>PubTabNet</p>
<p>FinTabNet</p>
<p>Pub'Tables-1M</p>
<p>OTSL</p>
<p>HTML</p>
<p>OTSL</p>
<p>HTML</p>
<p>OTSL</p>
<p>HTMLI</p>
<p>5.3</p>
<p>15.3 Qualitative Results</p>
<p>The OTS: vocabulary is comprised of the following tokens.</p>
<p>"C" cell - a new table cell that either has or does not have cell content</p>
<p>"L" cell - left-looking cell, merging with the left neighbor cell to create a</p>
<p>"U" cell - up-looking cell, merging with the upper neighbor cell to create a span</p>
<p>"X" cell - cross cell, to merge with both left and upper neighbor cells</p>
<p>"NL" - new-line, switch to the next row.</p>
<p>A notable attribute of OTSL is that it has the capability of achieving lossless</p>
<p>D</p>
<p>expression using a data frame called dif in R where you want to summarize x by month:....</p>
<p>require(plyr)</p>
<p>df &lt;- data. frame(|</p>
<p>x = runif(120, 1, 168),</p>
<p>2 = runif(120, 1.7, 20.7), y = runif(120, 7, 334).</p>
<p>month = rep(c(5,6,7,8),30), week = sample(1:4, 120, TRUE)</p>
<p>ddply(df, · (month, week), sumarize, sd = round(sd(x), 2))</p>
<p>‹picture&gt;&lt;loc_102&gt;&lt;loc_85&gt;&lt;loc_416&gt;&lt;loc_160&gt;~</p>
<p>&lt;flow_chart&gt;~</p>
<p>‹/picture&gt;~</p>
<p>&lt;otsl&gt;&lt;loc_63&gt;&lt;loc_78&gt;&lt;loc_436&gt;&lt;loc_158&gt;*|</p>
<p>‹ched›Data set. &lt;ched&gt;Language&lt;ched&gt;TEDs. &lt;Icel&gt;.</p>
<p>mean = round(mean(x), 2),</p>
<p>Figure 2: DocTags format describes key features of document elements: type of an element (text, picture, table, code, etc.), location on the page, and content. Nested tags convey additional information: pictures and tables may nest captions, table structure is represented in OTSL tags, lists nest list items, code and pictures carry a classification. All DocTags output is taken from SmolDocling predictions, artificial line-breaks and dots (...) were inserted for better readability. A, B, C - snippets of various pages from [55], D - from [92]</p>
<h2>3.1 Model Architecture</h2>
<p>The model architecture is illustrated in Figure 1. SmolVLM-256M relies on SigLIP base patch-16/512 (93M) as visual backbone and compared to the 2.2B version of the same model, its training data was rebalanced to emphasize document understanding (41%) and image captioning (14%), combining The Cauldron [42], Docmatix [41] datasets with the addition of MathWriting [23]. It uses lightweight variant of the SmolLM-2 family (135M) [2] as the language backbone and employs a radical pixel shuffle method that compresses each 512x512 image patch into 64 visual tokens. Last but not least, tokenization efficiency was also improved in SmolVLM-256M by increasing the pixel-to-token ratio to 4096 pixels per token and introducing special tokens for sub-image separators.</p>
<h2>3.2 DocTags</h2>
<p>To train VLMs for standardized document conversion, we introduce the DocTags markup format, illustrated in Figure 3. Inspired by OTSL [55], DocTags define a structured vocabulary of unambiguous tags and rules that explicitly separate textual content from document structure. This disentanglement improves performances of Image-to-Sequence models by minimizing confusions. In contrast, direct conversion to standard formats such as HTML or Markdown is often lossy and ambiguous, lacking explicit structural markup, failing to preserve page layout context, and increasing total tokens count. Pieces of text content that represent basic elements in DocTags are enclosed in XML-style tags. Tags define document block types such as: text, caption, footnote, formula, title, list_item, page_footer, page_header, picture, section_header, otsl, code, document_index, etc. Each element can nest additional location tags encoding its position on the page as a bounding box, represented in DocTags as &lt;loc_ x1 &gt;&lt;loc_ y1 &gt;&lt;loc_ x2 &gt;&lt;loc_ y2 &gt; .</p>
<p>Special blocks, such as tables and images, nest additional descriptors for captions, tabular structure, or image classes. To encode table structures, we fully include the OTSL vocabulary into DocTags . OTSL has the advantage of being minimalist and supporting cell spans and header information. Its tokens are interleaved with text, allowing</p>
<p>,&lt;ched&gt;mAP(0.75)&lt;ched&gt;Inference..&lt;nl&gt;~</p>
<p>SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion simultaneous encoding of both table structure and content. In order to promote robust visual-semantic alignment in our document understanding pipeline, we maintain a uniform DocTags representation for cropped page elements (e.g., tables, code, equations) that is identical to their full-page counterparts. This consistency ensures that the model can leverage similar visual cues whether an element is presented in isolation or in the context of an entire page, thereby strengthening the learning signal for both computer vision and downstream machine learning tasks.</p>
<p>For a complete list of tags and detailed description of the format, consult the appendix.</p>
<h2>3.3 Model Training</h2>
<p>We employ a curriculum learning approach to progressively align our model for document conversion, ensuring faster convergence. As an initial step, we incorporate DocTags as tokens in our tokenizer. To align the LLM part we freeze the vision encoder and train only the remaining network to adapt it to the new output format which it hasn't seen before. To ensure comprehensive coverage of all DocTags, we maintain a balanced mix of tasks and data types during training. Next, we unfreeze the vision encoder and train the model on our pretraining datasets (see Sec. 4.1), along with all task-specific conversion datasets, including tables, code, equations, and charts. Finally, we fine-tune using all available datasets.</p>
<h2>4 Data</h2>
<p>For document understanding tasks, each page image represents the visual layout of textual elements-such as paragraphs, formulas, tables, code, or figures with embedded textual content. Publicly available datasets that unify all these characteristics remain limited, and existing corpora often suffer from small sample sizes or fragmented formats. In this section, we explain how we mitigate these issues in both pre-training and task-specific training scenarios.</p>
<h2>4.1 Pre-training datasets</h2>
<p>In the domain of document understanding, currently available open-access document pre-training datasets include OCR-IDL [10], IIT-CDIP [44], CCpdf [83], and WordScape [88]. OCR-IDL and IIT-CDIP, industry-sourced from the 1990s, provide only page images and OCR text. CCpdf and WordScape, derived from Common Crawl, focus on raw PDFs and Word documents, respectively. While WordScape enriches documents with layout, table structure, and topic information via XML markup, its visual variability is limited. Notably, none of these datasets offer rich annotations for all page elements.</p>
<p>DocLayNet-PT. Given the lack of public multimodal document pre-training data with sufficient annotation features, we constructed a new document pre-training dataset named DocLayNet-PT. DocLayNet-PT is a 1.4M pages dataset extracted from the DocFM dataset [25] of unique PDF documents, sourced from CommonCrawl, Wikipedia, and business-related documents. We focused on this dataset to look for documents including visually distinct content such as equations, tables, code and charts with colorful layouts. This dataset was augmented with weak annotations through a series of processing steps. First, we applied PDF parsing to all pages to obtain the text layout including positional information using Docling [7, 53]. Next, we performed an enrichment step to obtain an enhanced, machine-friendly representation of each page. Specifically, we provide annotations for layout elements, table structure, language, topic, and figure classification on each page in DocTag format.</p>
<p>Docmatix. To preserve SmolVLM's original DocVQA capabilities, we adapted the same weak annotation strategy used for DocLayNet-PT and applied it to the 1.3M documents inside Docmatix dataset [41]. In addition to the 9.5 million DocVQA questions, we introduced another instruction requiring the full conversion of multi-page documents to DocTags.</p>
<h2>4.2 Task-specific Datasets</h2>
<p>Layout. To optimize prediction quality for document layout and table structure, we sampled 76K pages from DocLayNet-PT. Each page underwent human annotation and rigorous quality review, resulting in a high-quality ground-truth dataset. Of these, 60K pages were utilized for fine-tuning, constituting what we refer to as DocLayNet v2. Additionally, we extracted 63K pages from WordScape, specifically filtering for pages containing both text and tables, and utilized their inherently structure-preserving format as a reliable source of ground truth. To further enhance the model's adaptability to diverse layouts, colors, and fonts, we synthetically generated 250K pages using content from Wikipedia, creating the dataset known as SynthDocNet. This synthetic dataset enables the controlled generation of various document types with precise annotations. All datasets share a unified annotation format and class definitions, ensuring consistency and comprehensive document representation.</p>
<p>Tables. For the table structure recognition task we trained our model on public datasets such as PubTables-1M [78], FinTabNet [103], WikiTableSet [54] and tabular information extracted from WordScape documents. We transformed table structure information from the original ground-truth to OTSL format [55]. Then, we interleaved each cell tag with ground-truth text. This yields a condensed sequence which represents the structure of the table (including merged cells, and header declaration) along with its textual content.</p>
<p>Charts. For chart reconstruction, several datasets are publicly available [58, 33, 34, 93] that consist of both synthetic and web-scraped charts. However, these datasets are limited by either the quantity of real-world data or the visual diversity of synthetic examples. This has prompted recent studies [50, 26, 60, 94, 93, 13, 57] to crawl or render additional datasets, which unfortunately are rarely shared publicly. To address these gaps, we generated our own chart dataset comprising four types of charts: line, pie, bar, and stacked bar. Using data from 90,000 tables sourced from the FinTabNet dataset [103], we generated a comprehensive dataset containing 5,000 line charts, 380,000 pie charts, 380,000 bar charts, and 77,000 stacked bar charts. Each data point was rendered using three different visualization libraries (details provided in the Appendix), yielding a total of 2.5 million visually diverse charts.</p>
<p>Code. Technical books and scientific documents frequently include code snippets, making it essential for documentprocessing models to accurately interpret code structure, particularly indentation, which carries semantic significance in certain programming languages. Several publicly available datasets provide extensive collections of code snippets with permissive licenses in text format [38, 71], typically aggregated from sources like GitHub. However, a common limitation is that these datasets contain only text-based code snippets, lacking any image-based representation. To address this gap, we leveraged LaTeX and Pygments [16] to generate visually diverse renderings of code (details provided in the Appendix). Our dataset includes 9.3M million rendered code snippets generated at 120 dpi, covering 56 programming languages.</p>
<p>Equations. Accurately parsing mathematical formulas is crucial for document-processing models, as formulas often contain nuanced structures and semantics critical for scientific understanding. To effectively train our model for formula transcription, we combined publicly available datasets with a custom-rendered dataset. The public datasets [18, 22, 11, 23] collectively comprise approximately 730k unique formulas. In addition, we extracted 4.7 million formulas from arXiv by applying a regular expression to the source LaTeX code. The extracted formulas underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized LaTex code. Subsequently, these normalized equations were visually rendered using LaTeX (details provided in the Appendix). Overall, our dataset includes 5.5 million unique formulas, each rendered at a resolution of 120 dpi.</p>
<table><caption><div class="caption">Table 1: Datasets. Overview of the datasets used for the training and evaluation of SmolDocling. Datasets we contribute as part of this publication are denoted with (*).</div></caption><tbody><tr><th>Dataset Name</th><th>Size</th><th>Type</th></tr><tr><th>TheCauldron [42]</th><td>2.63M</td><td></td></tr><tr><th>DocLayNet-PT-Instruct</th><td>1.62M</td><td>Multi Task</td></tr><tr><th>DocMatix w/ Doc Conversion* [41]</th><td>952K</td><td></td></tr><tr><th>SynthChartNet*</th><td>2.5M</td><td></td></tr><tr><th>PlotQA [61]</th><td>225K</td><td></td></tr><tr><th>FigureQA [33]</th><td>100K</td><td>Chart</td></tr><tr><th>Unichart [57]</th><td>600K</td><td></td></tr><tr><th>ChartQA [58]</th><td>28K</td><td></td></tr><tr><th>DocLayNet-PT</th><td>1.40M</td><td></td></tr><tr><th>Doc4MLlaVa - RVLCDIP</th><td>742K</td><td></td></tr><tr><th>SynthDocNet</th><td>375K</td><td>Conversion</td></tr><tr><th>DocLayNet [70]</th><td>81K</td><td></td></tr><tr><th>DocLayNet v2</th><td>76K</td><td></td></tr><tr><th>WordScape-PT</th><td>63K</td><td></td></tr><tr><th>PubTable1M [78]</th><td>1M</td><td></td></tr><tr><th>WordScape Tables [88]</th><td>207K</td><td>Table</td></tr><tr><th>WikiTableSet (EN) [54]</th><td>204K</td><td></td></tr><tr><th>FinTabNet [103]</th><td>90K</td><td></td></tr><tr><th>SynthFormulaNet*</th><td>5.5M</td><td>Formula</td></tr><tr><th>SynthCodeNet*</th><td>9.3M</td><td>Code</td></tr></tbody></table>
<h2>4.3 Document Instruction Tuning datasets</h2>
<p>To reinforce the recognition of different page elements and introduce document-related features and no-code pipelines, we leveraged rule-based techniques and the Granite-3.1-2b-instruct [25] LLM. Using samples from DocLayNet-PT pages, we generated one instruction by randomly sampling layout elements from a page. These instructions included tasks such as 'Perform OCR at bbox,' 'Identify page element type at bbox,' and 'Extract all section headers from the page.' In addition, we train with Cauldron [42] to avoid catastrophic forgetting with the amount of conversation datasets introduced.</p>
<table><tbody><tr><th>Method</th><th>Size</th><th>Edit Distance ↓</th><th>F1-score ↑</th><th>Precision ↑</th><th>Recall ↑</th><th>BLEU ↑</th><th>METEOR ↑</th></tr><tr><th>Full-page</th><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><th>Qwen2.5 VL [9]</th><td>7B</td><td>0.56</td><td>0.72</td><td>0.80</td><td>0.70</td><td>0.46</td><td>0.57</td></tr><tr><th>GOT [89]</th><td>580M</td><td>0.61</td><td>0.69</td><td>0.71</td><td>0.73</td><td>0.48</td><td>0.59</td></tr><tr><th>Nougat (base) [12]</th><td>350M</td><td>0.62</td><td>0.66</td><td>0.72</td><td>0.67</td><td>0.44</td><td>0.54</td></tr><tr><th>SmolDocling (Ours)</th><td>256M</td><td>0.48</td><td>0.80</td><td>0.89</td><td>0.79</td><td>0.58</td><td>0.67</td></tr><tr><th>Code listings</th><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><th>SmolDocling (Ours)</th><td>256M</td><td>0.11</td><td>0.92</td><td>0.94</td><td>0.91</td><td>0.87</td><td>0.89</td></tr><tr><th>Equations</th><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><th>Qwen2.5 VL [9]</th><td>7B</td><td>0.22</td><td>0.89</td><td>0.91</td><td>0.87</td><td>0.68</td><td>0.77</td></tr><tr><th>GOT [89]</th><td>580M</td><td>0.11</td><td>0.95</td><td>0.95</td><td>0.96</td><td>0.85</td><td>0.91</td></tr><tr><th>Nougat (base) [12]</th><td>350M</td><td>0.62</td><td>0.60</td><td>0.60</td><td>0.53</td><td>0.33</td><td>0.41</td></tr><tr><th>SmolDocling (Ours)</th><td>256M</td><td>0.11</td><td>0.95</td><td>0.96</td><td>0.95</td><td>0.83</td><td>0.89</td></tr></tbody></table>
<p>Table 2: Structured document text recognition. We evaluate OCR performance and document formatting on DocLayNet, focusing on textual elements, excluding tables. Text accuracy is measured through multiple metrics (Edit Distance, F1-Score, Precision, Recall, BLEU and METEOR) as proposed in [12].</p>
<h2>5 Experiments</h2>
<p>In this section, we assess the performance of SmolDocling on document understanding tasks, and compare the results to previously published models and baselines. Capturing the qualities and defects of SmolDocling and other models in comparable numbers is a non-trivial effort for reasons we outline further below. Therefore, we report results on standard evaluation tasks and metrics independently (see section 5.2), but also provide a qualitative assessment of output characteristics (see section 5.3).</p>
<h2>5.1 Implementation Details</h2>
<p>Our training was conducted using 64 NVIDIA A100 80GB GPUs, with each epoch requiring 38 hours. The model was trained for a total of 4 epochs using the AdamW optimizer with a learning rate of 2 × 10 -4 and 2 × 10 -6 for the vision encoder once unfrozen. We also employed gradient clipping to 1.0 for stability and a warmup ratio of 0.03. For inference, using VLLM [39] on an A100 GPU, SmolDocling achieves a page conversion time of 0.35 seconds while occupying only 0.489 GB of VRAM. The model supports a maximum sequence length of 8,192 tokens and is trained to process up to three pages at a time.</p>
<h2>5.2 Quantitative Results</h2>
<p>To enable valid cross-model comparisons with SmolDocling, we addressed multiple dataset and harmonization challenges:</p>
<ul>
<li style="list-style-type: '· ';">Public evaluation datasets lack comprehensive multi-task annotations with diverse layouts. We augmented DocLayNet [70] with text content and reading-order to enable evaluation of both layout analysis and text recognition. Table structure, code listing, formula and chart recognition are evaluated on specific datasets.</li>
<li style="list-style-type: '· ';">Different models each follow their own conventions for output markup (e.g., Markdown, HTML, DocTags) and produce annotations which do not fully align in semantics. For example, SmolDocling is trained to produce 14 distinct layout-class tags via DocTags , while Qwen2.5-VL combines standard HTML tags and a few explicit class attributes with only partially compatible definitions. GOT and Nougat output Markdown, which can express paragraphs, lists and tables without location annotation.</li>
</ul>
<p>SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion</p>
<ul>
<li style="list-style-type: '· ';">Input resolution of page images can affect model performance, but is rarely specified in literature. We standardized at 144 DPI (dots per inch), which reproduces enough detail to be perfectly legible by humans while keeping resource demand for inference reasonable.</li>
</ul>
<p>Text Recognition (OCR). We first evaluate text recognition accuracy of SmolDocling, compared to Nougat, GOT and Qwen2.5-VL (Table 2), to verify accurate character transcription and reading order. The text-similarity metrics, adopted from [12] require plain formatting without non-textual elements. For different tasks, we conduct evaluations on diverse content types: full-page documents (from DocLayNet test set) in Markdown format, code snippets (from our SynthCodeNet dataset) in plaintext, and equations (from Im2Latex-230k [21]) in LaTeX format. For Qwen2.5-VL and SmolDocling, we convert HTML or DocTags output to Markdown while preserving formatting where possible.</p>
<p>Tables are omitted from the full-page output since they do not compare well through Markdown, due to loss of row and column spans. Code snippets must be reproduced with intact line breaks and indentation for accurate scoring. Formulas are evaluated separately in LaTeX format, which is natively produced by all the models tested in this work. The LaTeX outputs from all models, including the ground truth, undergo the same normalization procedure used for generating the SynthFormulaNet dataset. This ensures consistency and fairness in model comparisons.</p>
<p>SmolDocling significantly outperforms GOT, Nougat and also Qwen2.5-VL, which is 27 times larger, on every metric in full-page transcription (see Table 2. For the code parsing task, we report results exclusively for SmolDocling, as we introduce this task for the first time, and the other models were not explicitly trained for it. SmolDocling also demonstrates superior performance compared to Nougat and Qwen2.5-VL in the formula recognition task, closely matching GOT's results.</p>
<p>Layout. We evaluate element localization accuracy using the DocLayNet test set (Table 3), measuring the ability to reconstruct document representations that support content grounding and visual feature extraction. SmolDocling is compared against Qwen2.5-VL-7b [9], currently the only other VLM addressing this task. For a fair comparison, we map each model's label outputs and the ground-truth to six compatible classes: Text, Section Heading, List Item, Table, Picture, and Formula. Overall, SmolDocling outperforms Qwen2.5-VL-7b by a significant margin, however both models score far below a human baseline. This can be partially explained through inherent difficulty of the dataset, as indicated by relatively low human agreement scores. The scores are primarily impacted by moderate recall of element bounding boxes and different degrees of label confusion in both SmolDocling and Qwen2.5-VL. A majority of samples show accurate element bounding boxes as seen in the Appendix.</p>
<table><tbody><tr><td></td><th>SmolDocling</th><th>Qwen2.5-VL</th><th>Human</th></tr><tr><th>Formula</th><td>0.267</td><td>0.059</td><td>0.84</td></tr><tr><th>Table</th><td>0.266</td><td>0.262</td><td>0.79</td></tr><tr><th>List Item</th><td>0.296</td><td>0.090</td><td>0.87</td></tr><tr><th>Section Header</th><td>0.289</td><td>0.129</td><td>0.83</td></tr><tr><th>Text</th><td>0.204</td><td>0.180</td><td>0.85</td></tr><tr><th>Picture</th><td>0.066</td><td>0.078</td><td>0.70</td></tr><tr><th>Overall</th><td>0.231</td><td>0.133</td><td>0.82</td></tr></tbody></table>
<p>Table 3: Layout analysis comparison. Evaluation of layout analysis task on DocLayNet with a subset of 6 class labels in mAP[0.5:0.95] metric. Additional reference points are provided for an estimate of human performance [70].</p>
<p>Table Structure Recognition. We evaluate the accuracy of recovering table structures, i.e. shaping columns and rows with the correct text content and spans, through the TEDS metric [104], once including table text content and once on structure only with ommitted text (see Table 4). For isolated assessment, we provide cropped table images from FinTabNet [103] and PubTables-1M [78] test sets, which have poor image quality (72 dpi, compression artifacts). SmolDocling performs competitively against significantly larger models, despite challenges with text transcription from the low-resolution image crops, a limitation likely attributable to insufficient training at such resolutions. Scores for structure only TEDS evaluation are therefore much stronger.</p>
<p>Chart Extraction. In this task, we evaluate the ability to convert cropped sections of charts into datapoint tables. All table ouptut is transformed into HTML and compared against the ground-truth HTML using TEDS score, as reported in Table 5. We attribute SmolDocling's performance on charts to the variability in ground-truth styles and the differing interpretations of representations across datasets, which we observe leads to inconsistencies in the model's predictions. Despite its significantly smaller size, SmolDocling achieves a competitive TEDs score.</p>
<p>Overall, SmolDocling achieves leading performance across a wide range of document conversion tasks and outperforms significantly larger models in text recognition, layout analysis and structure extraction. Furthermore, the model's</p>
<p>SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion</p>
<table><tbody><tr><th>Method</th><th>Model Size</th><th>FinTabNet</th><th>PubTables-1M</th></tr><tr><th>SmolVLM [56]</th><td>2.2B</td><td>0.18</td><td>-</td></tr><tr><th>Granite Vision [81]</th><td>2.98B</td><td>0.54</td><td>-</td></tr><tr><th>TableFormer† [55]</th><td>52M</td><td>0.89</td><td>0.84</td></tr><tr><th>SmolDocling (Ours)</th><td>256M</td><td>0.52 (0.81)</td><td>0.65 (0.88)</td></tr></tbody></table>
<p>Table 4: Table structure and cell content reconstruction. Numbers are given in TEDS metric including text content, with structure-only score in brackets where available. We produced results for SmolDocling and TableFormer, other results are reflected here as published. †TableFormer extracts the content through external OCR.</p>
<table><tbody><tr><td>Model</td><th>Size</th><th>TEDs ↑</th></tr><tr><th>Phi-3.5-vision</th><td>4B</td><td>0.40</td></tr><tr><th>Granite Vision [81]</th><td>3B</td><td>0 . 95</td></tr><tr><th>SmolVLM [56]</th><td>2.2B</td><td>0.02</td></tr><tr><th>Molmo-E</th><td>1B</td><td>0.54</td></tr><tr><th>SmolDocling (Ours)</th><td>256M</td><td>0.75</td></tr></tbody></table>
<p>Table 5: Chart analysis. We compare SmolDocling to other small-sized VLMs on the chart-to-table task using the TEDS metric. The results in this table are based on the work reported in [25].</p>
<p>high performance in downstream tasks, such as code listing and formula recognition, demonstrates its versatility and real-world applicability. Notably, we establish benchmark results for the code listing recognition, setting a standard for future evaluations.</p>
<h2>5.3 Qualitative Results</h2>
<p>Several characteristics of SmolDocling output are not evident from task-based metrics alone. In this section, we highlight different output characteristics observed in practice. More illustrated samples can be found in the Appendix.</p>
<p>As presented in Figure 3, SmolDocling outputs a sequence of DocTags to encode content, structure and layout. A few notable traits beyond straight content reproduction include: 1) Labeling of page headers and footers, which allows ignoring these repetitive elements in downstream applications. 2) Linking of information, such as captions to tables or pictures, or list elements to parent list through nesting of tags. 3) Preserved line breaks and indentation for code snippets, while line wraps in paragraphs are correctly removed.</p>
<p>Among typical defects found in the output of SmolDocling are missing tags (e.g. absent location tags), malformed structure (such as missing closing tags), and endless repetition of tokens from some point in a page. Both of these phenomena are commonly seen in auto-regressive models and may prevent parsing or rendering the output correctly (see Appendix). Despite these limitations, SmolDocling demonstrates unique robustness advantages compared to ensemble systems like Docling, since conversion output is inferred in a single pass. In the latter, errors may accumulate throughout the ensemble model pipeline. For example, a mislocated table detection will lead to missing or distorted output of table structure and content, since this depends on correct table localization. In contrast, SmolDocling can reproduce the tables correctly, even with incorrectly determined location.</p>
<h2>6 Conclusion</h2>
<p>In this work, we introduce SmolDocling, an efficient and compact VLM optimized for document conversion while providing rich output representation. We also present a suite of new datasets with a unified format for document conversion, including the novel task of code listing transcription. We identify page element localization as a critical area requiring further refinement, where targeted techniques will significantly enhance performance in future iterations. Our results conclusively demonstrate that smaller models with unified, optimized output formats such as DocTags can effectively compete with significantly larger models, establishing a clear pathway for resource-efficient multi-task document understanding models.</p>
<h2>References</h2>
<ol>
<li style="list-style-type: '[1] ';">Grobid. https://github.com/kermitt2/grobid , 2008-2025.</li>
<li style="list-style-type: '[2] ';">Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf. SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model, 2025.</li>
<li style="list-style-type: '[3] ';">Amazon Web Services. Amazon Textract. https://aws.amazon.com/textract/ , 2024. Accessed: 2024-11-11.</li>
<li style="list-style-type: '[4] ';">Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403 , 2023.</li>
<li style="list-style-type: '[5] ';">Anthropic. Claude-3.5. https://www.anthropic.com/news/claude-3-5-sonnet , 2024. Accessed: 202402-11.</li>
<li style="list-style-type: '[6] ';">Christoph Auer, Michele Dolfi, André Carvalho, Cesar Berrospi Ramis, and Peter WJ Staar. Delivering Document Conversion as a Cloud Service with High Throughput and Responsiveness. In 2022 IEEE 15th International Conference on Cloud Computing (CLOUD) , pages 363-373. IEEE, 2022.</li>
<li style="list-style-type: '[7] ';">Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Nikolaos Livathinos, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Fabian Lindlbauer, Kasper Dinkla, et al. Docling Technical Report. arXiv preprint arXiv:2408.09869 , 2024.</li>
<li style="list-style-type: '[8] ';">Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 , 2023.</li>
<li style="list-style-type: '[9] ';">Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025.</li>
<li style="list-style-type: '[10] ';">Ali Furkan Biten, Rubèn Tito, Lluis Gomez, Ernest Valveny, and Dimosthenis Karatzas. Ocr-idl: Ocr annotations for industry document library dataset. In European Conference on Computer Vision , pages 241-252. Springer, 2022.</li>
<li style="list-style-type: '[11] ';">Lukas Blecher. LaTeX-OCR, 2024. Accessed: 2024-11-09.</li>
<li style="list-style-type: '[12] ';">Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint arXiv:2308.13418 , 2023.</li>
<li style="list-style-type: '[13] ';">Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. OneChart: Purify the Chart Structural Extraction via One Auxiliary Token. arXiv preprint arXiv:2404.09987 , 2024.</li>
<li style="list-style-type: '[14] ';">Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research , 25(70):1-53, 2024.</li>
<li style="list-style-type: '[15] ';">Pyecharts Contributors. pyecharts: Python library for Echarts, 2024. Accessed: 2024-11-06.</li>
<li style="list-style-type: '[16] ';">Pygments Contributors. Pygments: Python syntax highlighter, 2024. Accessed: 2024-11-06.</li>
<li style="list-style-type: '[17] ';">Brian Davis, Bryan Morse, Brian Price, Chris Tensmeyer, Curtis Wigington, and Vlad Morariu. End-to-end document recognition and understanding with dessurt. In European Conference on Computer Vision , pages 280-296. Springer, 2022.</li>
<li style="list-style-type: '[18] ';">Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M Rush. Image-to-markup generation with coarse-to-fine attention. In International Conference on Machine Learning , pages 980-989. PMLR, 2017.</li>
<li style="list-style-type: '[19] ';">Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model. arXiv preprint arXiv:2401.16420 , 2024.</li>
<li style="list-style-type: '[20] ';">Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Songyang Zhang, Haodong Duan, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Zhe Chen, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Kai Chen, Conghui He, Xingcheng Zhang, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD. arXiv preprint arXiv:2404.06512 , 2024.</li>
<li style="list-style-type: '[21] ';">Gregory Eritsyan. Im2LaTeX 230K. https://www.kaggle.com/datasets/gregoryeritsyan/ im2latex-230k , 2023. Accessed: 2025-03-06.</li>
<li style="list-style-type: '[22] ';">Hugging Face. latex-formulas Dataset, 2024. Accessed: 2024-11-09.</li>
<li style="list-style-type: '[23] ';">Philippe Gervais, Asya Fadeeva, and Andrii Maksai. Mathwriting: A dataset for handwritten mathematical expression recognition. URL https://arxiv. org/abs/2404.10690 , 2024.</li>
<li style="list-style-type: '[24] ';">Google Cloud. Document AI. https://cloud.google.com/document-ai , 2024. Accessed: 2024-11-11.</li>
<li style="list-style-type: '[25] ';">IBM Granite Team. Granite 3.0 Language Models, 2024.</li>
</ol>
<ol>
<li style="list-style-type: '[26] ';">Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483 , 2023.</li>
<li style="list-style-type: '[27] ';">Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403.12895 , 2024.</li>
<li style="list-style-type: '[28] ';">Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding. arXiv preprint arXiv:2409.03420 , 2024.</li>
<li style="list-style-type: '[29] ';">Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia , pages 4083-4091, 2022.</li>
<li style="list-style-type: '[30] ';">J. D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science &amp; Engineering , 9(3):90-95, 2007.</li>
<li style="list-style-type: '[31] ';">Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) , volume 2, pages 1-6. IEEE, 2019.</li>
<li style="list-style-type: '[32] ';">Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 , 2024.</li>
<li style="list-style-type: '[33] ';">Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300 , 2017.</li>
<li style="list-style-type: '[34] ';">Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. arXiv preprint arXiv:2203.06486 , 2022.</li>
<li style="list-style-type: '[35] ';">Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Donut: Document understanding transformer without ocr. arXiv preprint arXiv:2111.15664 , 7(15):2, 2021.</li>
<li style="list-style-type: '[36] ';">Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton. PubChem 2019 update: improved access to chemical data. Nucleic Acids Research , 47(D1):D1102-D1109, 10 2018.</li>
<li style="list-style-type: '[37] ';">Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015-4026, 2023.</li>
<li style="list-style-type: '[38] ';">Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. The Stack: 3 TB of permissively licensed source code. Preprint , 2022.</li>
<li style="list-style-type: '[39] ';">Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023.</li>
<li style="list-style-type: '[40] ';">Greg Landrum. RDKit: Open-Source Cheminformatics Software. http://www.rdkit.org/. Accessed: 1 January 2023.</li>
<li style="list-style-type: '[41] ';">Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and better understanding vision-language models: insights and future directions., 2024.</li>
<li style="list-style-type: '[42] ';">Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models?, 2024.</li>
<li style="list-style-type: '[43] ';">Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Conference on Machine Learning , pages 18893-18912. PMLR, 2023.</li>
<li style="list-style-type: '[44] ';">David Lewis, Gady Agam, Shlomo Argamon, Ophir Frieder, David Grossman, and Jefferson Heard. Building a test collection for complex document information processing. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval , pages 665-666, 2006.</li>
<li style="list-style-type: '[45] ';">Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 , 2024.</li>
<li style="list-style-type: '[46] ';">Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895 , 2024.</li>
<li style="list-style-type: '[47] ';">Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.</li>
<li style="list-style-type: '[48] ';">Han Lin, Peng Yang, and Fanlong Zhang. Review of scene text detection and recognition. Archives of computational methods in engineering , 27(2):433-454, April 2020.</li>
<li style="list-style-type: '[49] ';">Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. arXiv preprint arXiv:2212.10505 , 2022.</li>
</ol>
<ol>
<li style="list-style-type: '[50] ';">Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. arXiv preprint arXiv:2212.09662 , 2022.</li>
<li style="list-style-type: '[51] ';">Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved Baselines with Visual Instruction Tuning, 2023.</li>
<li style="list-style-type: '[52] ';">Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. In NeurIPS , 2023.</li>
<li style="list-style-type: '[53] ';">Nikolaos Livathinos, Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Kasper Dinkla, Yusik Kim, Shubham Gupta, Rafael Teixeira de Lima, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, and Peter W. J. Staar. Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion, 2025.</li>
<li style="list-style-type: '[54] ';">Nam Ly., Atsuhiro Takasu., Phuc Nguyen., and Hideaki Takeda. Rethinking Image-Based Table Recognition Using Weakly Supervised Methods. pages 872-880, 2023.</li>
<li style="list-style-type: '[55] ';">Maksym Lysak, Ahmed Nassar, Nikolaos Livathinos, Christoph Auer, and Peter Staar. Optimized Table Tokenization for Table Structure Recognition. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 37-50, Cham, 2023. Springer Nature Switzerland.</li>
<li style="list-style-type: '[56] ';">Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf. SmolVLM: Redefining small and efficient multimodal models. 2025.</li>
<li style="list-style-type: '[57] ';">Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arXiv:2305.14761 , 2023.</li>
<li style="list-style-type: '[58] ';">Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244 , 2022.</li>
<li style="list-style-type: '[59] ';">Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 2200-2209, 2021.</li>
<li style="list-style-type: '[60] ';">Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. Chartassisstant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. arXiv preprint arXiv:2401.02384 , 2024.</li>
<li style="list-style-type: '[61] ';">Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. PlotQA: Reasoning over Scientific Plots. In The IEEE Winter Conference on Applications of Computer Vision (WACV) , March 2020.</li>
<li style="list-style-type: '[62] ';">Microsoft Azure. AI Document Intelligence. https://azure.microsoft.com/en-us/products/ ai-services/ai-document-intelligence , 2024. Accessed: 2024-11-11.</li>
<li style="list-style-type: '[63] ';">Lucas Morin, Martin Danelljan, Maria Isabel Agea, Ahmed Nassar, Valery Weber, Ingmar Meijer, Peter Staar, and Fisher Yu. MolGrapher: Graph-based Visual Recognition of Chemical Structures. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 19552-19561, October 2023.</li>
<li style="list-style-type: '[64] ';">Lucas Morin, Valéry Weber, Gerhard Ingmar Meijer, Fisher Yu, and Peter W. J. Staar. PatCID: an open-access dataset of chemical structures in patent documents. Nature Communications , 15(1):6532, Aug 2024.</li>
<li style="list-style-type: '[65] ';">OpenAI. Hello GPT-4O. https://openai.com/index/hello-gpt-4o/ , 2024. Accessed: 2024-02-09, 2024-0211, 2024-02-12.</li>
<li style="list-style-type: '[66] ';">R OpenAI. Gpt-4 technical report. arxiv 2303.08774. View in Article , 2(5), 2023.</li>
<li style="list-style-type: '[67] ';">Vik Paruchuri. Marker: Convert PDF to Markdown Quickly with High Accuracy. https://github.com/VikParuchuri/marker, 2024.</li>
<li style="list-style-type: '[68] ';">Vik Paruchuri. Texify. https://github.com/VikParuchuri/texify , 2024. Accessed: 2024-11-11.</li>
<li style="list-style-type: '[69] ';">Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824 , 2023.</li>
<li style="list-style-type: '[70] ';">Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S. Nassar, and Peter Staar. DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 3743-3751, New York, NY, USA, 2022. Association for Computing Machinery.</li>
<li style="list-style-type: '[71] ';">Ruchir Puri, David Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, Shyam Ramji, Ulrich Finkler, Susan Malaika, and Frederick Reiss. CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks, 2021.</li>
<li style="list-style-type: '[72] ';">Edward O. Pyzer-Knapp, Matteo Manica, Peter Staar, Lucas Morin, Patrick Ruch, Teodoro Laino, John R. Smith, and Alessandro Curioni. Foundation models for materials discovery - current state and future directions. npj Computational Materials , 11(1):61, Mar 2025.</li>
<li style="list-style-type: '[73] ';">Yujie Qian, Jiang Guo, Zhengkai Tu, Zhening Li, Connor W. Coley, and Regina Barzilay. MolScribe: Robust Molecular Structure Recognition with Image-to-Graph Generation. Journal of Chemical Information and Modeling , 63(7):1925-1934, Apr 2023.</li>
</ol>
<ol>
<li style="list-style-type: '[74] ';">Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research , 21(140):1-67, 2020.</li>
<li style="list-style-type: '[75] ';">Kohulan Rajan, Henning Otto Brinkhaus, M. Isabel Agea, Achim Zielesny, and Christoph Steinbeck. DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications. Nature Communications , 14(1):5045, Aug 2023.</li>
<li style="list-style-type: '[76] ';">Johannes Rausch, Octavio Martinez, Fabian Bissig, Ce Zhang, and Stefan Feuerriegel. Docparser: Hierarchical document structure parsing from renderings. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 4328-4338, 2021.</li>
<li style="list-style-type: '[77] ';">Felix M Schmitt-Koopmann, Elaine M Huang, Hans-Peter Hutter, Thilo Stadelmann, and Alireza Darvishy. MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition. IEEE Access , 2024.</li>
<li style="list-style-type: '[78] ';">Brandon Smock, Rohith Pesala, and Robin Abraham. PubTables-1M: Towards comprehensive table extraction from unstructured documents. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4624-4632, 2022.</li>
<li style="list-style-type: '[79] ';">Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mohit Bansal. Unifying vision, text, and layout for universal document processing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19254-19264, 2023.</li>
<li style="list-style-type: '[80] ';">Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.</li>
<li style="list-style-type: '[81] ';">Granite Vision Team, Leonid Karlinsky, Assaf Arbelle, Abraham Daniels, Ahmed Nassar, Amit Alfassi, Bo Wu, Eli Schwartz, Dhiraj Joshi, Jovana Kondic, Nimrod Shabtay, Pengyuan Li, Roei Herzig, Shafiq Abedin, Shaked Perek, Sivan Harary, Udi Barzelay, Adi Raz Goldfarb, Aude Oliva, Ben Wieles, Bishwaranjan Bhattacharjee, Brandon Huang, Christoph Auer, Dan Gutfreund, David Beymer, David Wood, Hilde Kuehne, Jacob Hansen, Joseph Shtok, Ken Wong, Luis Angel Bathen, Mayank Mishra, Maksym Lysak, Michele Dolfi, Mikhail Yurochkin, Nikolaos Livathinos, Nimrod Harel, Ophir Azulai, Oshri Naparstek, Rafael Teixeira de Lima, Rameswar Panda, Sivan Doveh, Shubham Gupta, Subhro Das, Syed Zawad, Yusik Kim, Zexue He, Alexander Brooks, Gabe Goodhart, Anita Govindjee, Derek Leist, Ibrahim Ibrahim, Aya Soffer, David Cox, Kate Soule, Luis Lastras, Nirmit Desai, Shila Ofek-koifman, Sriram Raghavan, Tanveer Syeda-Mahmood, Peter Staar, Tal Drory, and Rogerio Feris. Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence, 2025.</li>
<li style="list-style-type: '[82] ';">Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.</li>
<li style="list-style-type: '[83] ';">Michał Turski, Tomasz Stanisławek, Karol Kaczmarek, Paweł Dyda, and Filip Graliński. CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 348-365, Cham, 2023. Springer Nature Switzerland.</li>
<li style="list-style-type: '[84] ';">Unstructured.io Team. Unstructured.io: Open-Source Pre-Processing Tools for Unstructured Data. https://unstructured.io, 2024. Accessed: 2024-11-19.</li>
<li style="list-style-type: '[85] ';">Vicuna. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. https://vicuna.lmsys. org/ , 2023.</li>
<li style="list-style-type: '[86] ';">Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. MinerU: An Open-Source Solution for Precise Document Content Extraction, 2024.</li>
<li style="list-style-type: '[87] ';">Michael L Waskom. Seaborn: statistical data visualization. Journal of Open Source Software , 6(60):3021, 2021.</li>
<li style="list-style-type: '[88] ';">Maurice Weber, Carlo Siebenschuh, Rory Butler, Anton Alexandrov, Valdemar Thanner, Georgios Tsolakis, Haris Jabbar, Ian Foster, Bo Li, Rick Stevens, et al. WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data. Advances in Neural Information Processing Systems , 36, 2024.</li>
<li style="list-style-type: '[89] ';">Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model. arXiv preprint arXiv:2409.01704 , 2024.</li>
<li style="list-style-type: '[90] ';">Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824-24837, 2022.</li>
<li style="list-style-type: '[91] ';">David Weininger. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences , 28(1):31-36, Feb 1988.</li>
<li style="list-style-type: '[92] ';">Wes McKinney and the Pandas Development Team. Pandas: powerful Python data analysis toolkit, Release 1.4.4. https: //pandas.pydata.org/pandas-docs/version/1.4/pandas.pdf , 2022.</li>
<li style="list-style-type: '[93] ';">Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Junchi Yan, and Yu Qiao. StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding. arXiv preprint arXiv:2309.11268 , 2023.</li>
</ol>
<ol>
<li style="list-style-type: '[94] ';">Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, et al. Chartx &amp; chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185 , 2024.</li>
<li style="list-style-type: '[95] ';">Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, et al. Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 , 2020.</li>
<li style="list-style-type: '[96] ';">Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery &amp; data mining , pages 1192-1200, 2020.</li>
<li style="list-style-type: '[97] ';">Zuoyu Yan, Xiaode Zhang, Liangcai Gao, Ke Yuan, and Zhi Tang. ConvMath: a convolutional sequence network for mathematical expression recognition. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 4566-4572. IEEE, 2021.</li>
<li style="list-style-type: '[98] ';">Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. arXiv preprint arXiv:2310.05126 , 2023.</li>
<li style="list-style-type: '[99] ';">Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition. arXiv preprint arXiv:2309.15112 , 2023.</li>
<li style="list-style-type: '[100] ';">Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLMXComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output. arXiv preprint arXiv:2407.03320 , 2024.</li>
<li style="list-style-type: '[101] ';">Qiming Zhang, Jing Zhang, Yufei Xu, and Dacheng Tao. Vision Transformer with Quadrangle Attention. arXiv preprint arXiv:2303.15105 , 2023.</li>
<li style="list-style-type: '[102] ';">Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. URL https://arxiv. org/abs/2205.01068 , 3:19-0, 2023.</li>
<li style="list-style-type: '[103] ';">Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 697-706, 2021.</li>
<li style="list-style-type: '[104] ';">Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes. Image-based table recognition: data, model, and evaluation. arXiv preprint arXiv:1911.10683 , 2019.</li>
<li style="list-style-type: '[105] ';">Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 , 2023.</li>
</ol>
<p>Multi Task</p>
<p>PubTablelM</p>
<p>SynthFormulaNet</p>
<figure><figcaption><div class="caption">Figure 3: Training datasets. A treemap visualization of dataset sizes contributing to the training of SmolDocling, categorized by type. Each rectangle represents a dataset, with its area proportional to the total dataset size. Each color indicates a dataset type. &#x27;Multi-Task&quot; refers to datasets that include multiple different tasks.</div></figcaption></figure>
<h2>A Additional Datasets Details</h2>
<h2>A.1 SynthChartNet</h2>
<p>SynthChartNet includes four distinct types of charts: line, pie, bar, and stacked bar, generated using three visualization libraries: Matplotlib [30], Seaborn [87], and Pyecharts [15]. Each data set was rendered through all three libraries, producing a final dataset of 2.5 million visually diverse charts. The rendering process was designed to leverage the full range of features available in these libraries, ensuring visual variety while maintaining the plausibility of the data presented. Figure 4 depicts 10 samples from SynthChartNet.</p>
<h2>A.2 SynthCodeNet</h2>
<p>To render the code snippets of SynthCodeNet, we employed two rendering libraries: LaTeX and Pygments [16]. LaTex, in conjunction with its listings package, enabled the generation of code renderings similar to those commonly seen in academic and technical publications, while Pygments facilitated renderings with aesthetics resembling those of IDEs. The rendering process utilized the complete set of features provided by these libraries to enhance visual diversity while preserving the plausibility of the presented data. This included randomizing various stylistic parameters such as font, font size, style, weight, line spacing, tab size, background color, line numbering (including randomizing the starting line number), and programming language-specific syntax highlighting. Figure 5 shows 4 samples from SynthCodeNet.</p>
<h2>A.3 SynthFormulaNet</h2>
<p>The extracted formulas of SynthFormulaNet underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized code. The normalization process consisted of the following major steps:</p>
<ol>
<li style="list-style-type: '1. ';">Token Filtering and Removal : Unnecessary or redundant LaTeX tokens were removed or replaced based on predefined policies. This included:</li>
<li style="list-style-type: '· ';">Eliminating unwanted tokens.</li>
<li style="list-style-type: '· ';">Replacing tokens with equivalent ones for consistency.</li>
</ol>
<p>ChartQA</p>
<p>MolGrapher Synthetic 300k WordScape-PocLayNe[nichart</p>
<p>PlotQA</p>
<p>SynthDocNet</p>
<p>FinTabNet</p>
<p>600</p>
<p>500 -</p>
<p>400-</p>
<p>300 -</p>
<p>200</p>
<p>100-</p>
<p>1700</p>
<p>borond suel</p>
<p>1800</p>
<p>1900</p>
<p>Property plant and equipment -</p>
<p>Iccumulated depre..</p>
<p>Cross asset volur</p>
<p>Construction in progress in millions except per share</p>
<p>Total revenues</p>
<p>Income (loss) from continuing</p>
<p>Income from continuing</p>
<p>Net income (loss)</p>
<p>Total assets</p>
<p>Short-term debt</p>
<p>Total debt (including capital</p>
<p>Total stockholders' equity</p>
<p>2000</p>
<p>Enteray Texal</p>
<p>-** nteray New or eani</p>
<p>Enteray Hississippl</p>
<p>Interay Art.</p>
<p>2100</p>
<p>2200</p>
<p>-Ind</p>
<p>2300</p>
<p>12.31</p>
<p>band improvements</p>
<p>Duldings and building</p>
<p>- machinery and equipmen</p>
<p>13.85</p>
<p>16.92</p>
<p>Total Cash Dividends Paid</p>
<p>Toral Cash Widends Paid der Common Share</p>
<p>2011</p>
<p>2012</p>
<p>2013</p>
<p>2014</p>
<p>Total unrecognized</p>
<p>2500 5000 7500 10000 12500 15000 17500 20000</p>
<p>1Year</p>
<p>United States loYear</p>
<p>· 5Year|</p>
<p>O..................... . ... . . ... ..... ........ .... ·</p>
<p>2075</p>
<p>2100</p>
<p>2125</p>
<p>2150</p>
<p>2175</p>
<p>2200</p>
<p>2225</p>
<p>First econd</p>
<p>Third</p>
<p>7.69</p>
<figure><figcaption><div class="caption">Figure 4: Samples from SynthChartNet.</div></figcaption></figure>
<p>24.7%</p>
<p>2250</p>
<p>25.3%</p>
<p>· 3Year</p>
<p>24.4%</p>
<p>25.7%</p>
<p>16..</p>
<p>20.I</p>
<p>#pragma once</p>
<p>#include &lt;sdm/world/solvable_by_dp. hpp&gt;</p>
<p>#include &lt;sdm/utils/value_function/initializer.hpp&gt;</p>
<p>namespace sdm namespace</p>
<p>{</p>
<p>class</p>
<p>{</p>
<p>registry protected:</p>
<p>typedef std:: map&lt;std:: string, std:: shared</p>
<p>_ptr&lt;</p>
<p>world, Config config) &gt; map_type;</p>
<p>Initializer &gt; (*) (std:: shared_ptr&lt;SolvableByDP&gt;</p>
<p>static map_type container;</p>
<p>initializer sane: Stage the app</p>
<p>A20RE, NEBAT NAVE: Y0218431-T5E-APP...</p>
<p>................</p>
<p>1t: contains (gathub. event.pull_request labela.*.name, 'stage*)</p>
<p>steps:</p>
<p>- uses: actions/checkout (vl</p>
<p>- name: pm install and build webpack run:</p>
<p>npm instal.</p>
<p>- uses: actions/upload-artifactênain apm run build</p>
<p>with:</p>
<p>path: public/</p>
<p>name: webpack artifacts</p>
<p>Busld-Docker-Image:</p>
<p>runs-on: ubuntu-latest</p>
<pre><code>public: static std:: vector<std:: string > available (): static std:: shared_ptr<Initializer > make(std :: string name, std:: shared_ptr<SolvableByDP> world, Config config = {}); } :1 template <class TInitializer > std:: shared_ptr<Initializer > createlnstance (std:: shared_ptr<SolvableByDP> world, Config config) return std:: make_shared<TInitializer >(world, config): package br.com.biot. integra caopagarmeapi.modulos.jwt.utids import br.com.biot. integracaopagarmeapi.config.exception.ValidacaoExceptipn import io.jsonwebtoken.Claims import lombok.extern.slf4j.Slf4j public class JwtCampoUtil{ public static String getCampold(Claims claims) {| tryi return (String) claims.get("id"); loge to tro do tentar recuperar I do JWT: 0'; claims toString): throw new ValidacaoException("Erro ao tentar recuperar o campo ID do Token."): public static String getCampo(String campo, Claims claims) { return (String) claims.get(campo); sagetro euro de entar recuperar o campo f] do IT: 0', campo, claims toString): throw new ValidacaoException( String.format("Erro ao tentar recuperar o campo %s do Token.", campo) 29| needs: build name: Build image and store in GitHub Packages steps: - name: Checkout uses: actions/checkout iva - name: Download built artifact uses: actions/download-arts factinain path: public - name: create inage and store in Packages uses: nattdavsa0351/actions/docker-gpr01.3.0 with: repo-token: 3{ (secrets.GITHUB,TOKEN) ) runneone wountuel tent needs: Build-Docker-Image name: Deploy app container to Azure steps: - name: "Login via Azure CLI" MIRA creda: 91{ SeCretA.AZURE_CREDENTIALS 1) - uses: azure/docker-logan@va with: logan-server: #{(env. IMAGE_REGISTRY_URL)) def main (filename, days) : · array I of buckets 0-8, each representing with that counter value fish_by_counter = [O for i in range (9)] for counter in read_counters (filename): fish_by_counter [counter) += 1 the number of fish for i in range (days): fish_by_counter · spawn (fish_by_counter) print (f"After {days) days: {sum(fish_by_counter)} fish") def spawn (fish_by_counter): # rotate left res = fish_by_counter (1:) + fish_by_counter [:1] # fish that were in bucket O are now new fish in bucket 8. restore the original fish into bucket 6. res [6] += res [8] return res def read_counters (filename) : with open (filename) as £ : return [int (counter) split (',')] for counter in f. readline ().strip(). -_name.. == '__main_.': main('input.txt', 80) main ('input.txt', 256)</code></pre>
<p>kejnKm kejnKm</p>
<p>kejnKm</p>
<p>=I+II + II.</p>
<div class="formula-not-decoded">Formula not decoded</div>
<p>d ≤ 1</p>
<p>Y</p>
<p>4.30</p>
<p>4.31</p>
<p>4.32</p>
<div class="formula-not-decoded">Formula not decoded</div>
<div class="formula-not-decoded">Formula not decoded</div>
<div class="formula-not-decoded">Formula not decoded</div>
<p>(x)</p>
<div class="formula-not-decoded">Formula not decoded</div>
<div class="formula-not-decoded">Formula not decoded</div>
<p>Figure 6: Samples from SynthFormulaNet.</p>
<p>SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion</p>
<ol>
<li style="list-style-type: '2. ';">Structural Normalization : The equation structure was standardized by:</li>
<li style="list-style-type: '· ';">Normalizing spacing between tokens.</li>
<li style="list-style-type: '· ';">Ensuring consistent use of tokens that require one or two braces.</li>
<li style="list-style-type: '· ';">Standardizing left and right delimiters.</li>
<li style="list-style-type: '3. ';">Token Simplification : Redundant tokens and patterns were simplified, including:</li>
<li style="list-style-type: '· ';">Collapsing consecutive redundant tokens (e.g., repeated primes, dots, or excessive spacing).</li>
<li style="list-style-type: '· ';">Converting special constructs such as \Big into \left and \right for consistency.</li>
</ol>
<p>The extracted equations were subsequently rendered using LaTex at 120 dpi. Several aspects of the rendering procedure have been randomized to ensure the visual diversity of the dataset. Each formula was rendered using a randomly selected font from a set of 85 possible options, with variations in font size, style, weight, and line spacing. Additionally, with a certain probability, each formula was enclosed within an "align" environment, allowing for the inclusion of a randomly generated equation number on either the left or right side of the equation. The equation number itself was chosen at random from a diverse set of possibilities, including small and large numbers, integers, floating-point values, or letters. Furthermore, it could be enclosed in single or double parentheses, brackets, or curly braces and was sometimes preceded by text labels such as 'Eqn.", 'Eqn", 'Eq.", or 'Equation". Figure 6 depicts 6 samples from SynthFormulaNet.</p>
<h2>A.4 DocTags vocabulary</h2>
<p>Here we provide a reference to a complete vocabulary of DocTags for SmolDocling , and their meaning.</p>
<p>We represent tags in an XML-like style of notation. Some tags have an opening and closing part, and wrap around textual content (or other tags) for example: &lt;text&gt;hello world&lt;/text&gt; . Other tags are standalone and do not have a closing tag; they mark a special instruction, for example: &lt;page_break&gt;</p>
<p>The complete DocTags snippet can represent a page or multiple pages, wrapped into &lt;doctag&gt;...&lt;/doctag&gt; . When content represents multiple pages, we use &lt;page_break&gt; tag as a separator.</p>
<p>Within the scope of &lt;doctag&gt; we place high-level, tags that wrap around the textual content of certain document blocks and identify the type of the block. These tags are: &lt;text&gt; , &lt;caption&gt; , &lt;footnote&gt; , &lt;formula&gt; , &lt;title&gt; , &lt;page_footer&gt; , &lt;page_header&gt; , &lt;picture&gt; , &lt;section_header&gt; , &lt;document_index&gt; , &lt;code&gt; , &lt;otsl&gt; , &lt;list_item&gt; , &lt;ordered_list&gt; , &lt;unordered_list&gt;</p>
<p>Each element can nest additional standalone location tags that encode its position on the page as a bounding box, represented in DocTags as &lt;loc_ x1 &gt;&lt;loc_ y1 &gt;&lt;loc_ x2 &gt;&lt;loc_ y2 &gt; . Every x,y pair of coordinates belongs to a fixed grid. In our case we used integer values in the range from 0 to 500 which proportionally maps to width and height of the page.</p>
<p>&lt;otsl&gt; element contains table representation with following OTSL tags, that are standalone and used to describe structure of a table: &lt;fcel&gt; , &lt;ecel&gt; , &lt;lcel&gt; , &lt;ucel&gt; , &lt;xcel&gt; , &lt;nl&gt; . These tags follow rules as described in [55], we additionally differentiate OTSL notion of a cell into &lt;fcel&gt; - full cell, or cell which contain content, and &lt;ecel&gt; - empty cell. We also augment tabular structure information with extra tags &lt;ched&gt; , &lt;rhed&gt; , &lt;srow&gt; that replace &lt;fcel&gt; to describe cells which belong to column and row headers of the table, as well as table section accordingly. We mark such headers when ground truth about table headers from the underlying dataset is available.</p>
<p>&lt;list_item&gt; elements are placed within &lt;ordered_list&gt; or &lt;unordered_list&gt; and define weather it's enumerated (ordered) list or not.</p>
<p>&lt;picture&gt; and &lt;otsl&gt; elements by themselves can encapsulate &lt;caption&gt; tag if appropriate picture or table has it's own caption. This way we can connect extra descriptive information to illustrations and tables in the document.</p>
<p>&lt;code&gt; elements contain pieces of code as a content, and as such, respect tabulation and line breaks. Code elements also contains special standalone classification tag -&lt;_ programming-language _&gt; where programminglanguage is an appropriate programming language name. Supported values (57 in total) for programming language names are: Ada , Awk , Bash , bc , C , C# , C++ , CMake , COBOL , CSS , Ceylon , Clojure , Crystal , Cuda , Cython , D , Dart , dc , Dockerfile , Elixir , Erlang , FORTRAN , Forth , Go , HTML , Haskell , Haxe , Java , JavaScript , Julia , Kotlin , Lisp , Lua , Matlab , MoonScript , Nim , OCaml , ObjectiveC , Octave , PHP , Pascal , Perl , Prolog , Python , Racket , Ruby , Rust , SML , SQL , Scala , Scheme , Swift , TypeScript , unknown , VisualBasic , XML , YAML</p>
<p>Within &lt;picture&gt; elements we include picture classification with extra standalone tags &lt; image_class&gt; . We classify images from our datasets into following categories: natural_image , pie_chart , bar_chart , line_chart ,</p>
<p>flow_chart , scatter_chart , heatmap , remote_sensing , chemistry_molecular_structure , chemistry_markush_structure , icon , logo , signature , stamp , qr_code , bar_code , screenshot , map , stratigraphic_chart , cad_drawing , electrical_diagram</p>
<h2>B Additional Details about the SmolDocling</h2>
<table><caption><div class="caption">Table 6: Supported Conversion Instructions.</div></caption><tbody><tr><th>Instruction</th><th>Description</th></tr><tr><td>Full conversion</td><td>Convert this page to docling.</td></tr><tr><td>Chart</td><td>Convert chart to table (e.g., &lt;chart&gt; ).</td></tr><tr><td>Formula</td><td>Convert formula to LaTeX (e.g., &lt;formula&gt; ).</td></tr><tr><td>Code</td><td>Convert code to text (e.g., &lt;code&gt; ).</td></tr><tr><td>Table</td><td>Convert table to OTSL (e.g., &lt;otsl&gt; ).</td></tr><tr><td rowspan="4">No-Code Ac- tions/Pipelines</td><td>OCR the text in a specific location: &lt;loc_155&gt;&lt;loc_233&gt;&lt;loc_206&gt;&lt;loc_237&gt;</td></tr><tr><td>Identify element at: &lt;loc_247&gt;&lt;loc_482&gt;&lt;loc_252&gt;&lt;loc_486&gt;</td></tr><tr><td>Find all &#x27;text&#x27; elements on the page, retrieve all section headers.</td></tr><tr><td>Detect footer elements on the page.</td></tr></tbody></table>
<p>Input Image</p>
<p>OH</p>
<p>Cl</p>
<p>· 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR</p>
<p>2.5-VL-72B and SmolDocling on a simple molecule image.</p>
<p>GOT-OCR 2.0</p>
<p>NH,</p>
<figure><figcaption><div class="caption">Figure 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR 2.0, Qwen2.5-VL-72B and SmolDocling on a simple molecule image.</div></figcaption></figure>
<h2>C Additional Qualitative Analysis</h2>
<h2>C.1 Molecule Image Recognition Experiment</h2>
<p>Extracting molecule images from documents has potential to accelerate research in chemistry and materials discovery [72]. Some recent document understanding models, such as GOT-OCR 2.0 [89] and Qwen2.5-VL [9], claim to perform molecule image recognition in documents. Following this direction, we experimented with training SmolDocling for molecule image recognition on full document pages and cropped images. In order to predict molecule structures (graphs) with SmolDocling, we use the molecule string identifier named SMILES [91].</p>
<p>Datasets. For training on cropped images, we use the MolGrapher-Synthetic-300K dataset [63]. The dataset contains 300K synthetic samples. It is created using real chemical-structures retrieved from the database PubChem [36] which are then synthetically drawn using the library RDKit [40]. For training of full pages, we use Doc2Mol-80K, a subset of the PatCID dataset [64]. Our subset contains 80 000 real patent documents published in the United-States between 2022 and 2024.</p>
<p>Evaluation. Figure 7 shows an example of predicted molecules for a simple molecule image. While some models capture high-level molecular features, they consistently fail to reconstruct its detailed structure. Notably, this molecule is trivial for specialized models such as MolGrapher [63], MolScribe [73] or DECIMER [75], which accurately recognize its structure. Reducing the gap between general document understanding models and specialized models may require incorporating a specialized vocabulary for SMILES tokens, explicitly encoding atoms and bonds, and extending the SMILES sequence with localization tokens for each atom.</p>
<h2>C.2 Layout analysis samples</h2>
<p>To illustrate and compare the visual grounding capabilities of SmolDocling, a set of sample predictions with SmolDocling and Qwen2.5-VL from DocLayNet [70] is shown in Table 7. Note that the element location results are independent from the correct reproduction of document content and structure.</p>
<p>Ground-Truth</p>
<p>46"</p>
<p>peR, heath and safety</p>
<p>Final gonn so yoR and convertible loan</p>
<p>tor save did exit) in the Opoon promp pur puse the Enter key:</p>
<p>trollowing display appears.</p>
<p>apposoices, tress later, yange Sample form nurter 3 in the Tet "description' prompt.</p>
<p>Hollowing display appears</p>
<p>Tyle doices, tress toler</p>
<p>Create earlay..</p>
<p>Moneyle Actior: Press the Enter key.</p>
<p>310.000,000 pal during the 2015</p>
<p>Sign untain Change in matters suusequent to</p>
<p>the end of the financial year</p>
<p>40</p>
<p>SmolDocling</p>
<p>2wen2.5-VL</p>
<p>Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. Win following, display appears we want yourse</p>
<p>Mum: F4 To 11S</p>
<p>Ground-Truth</p>
<p>301 Workers engarod in cential oe emeriency works pass ntial or emergency works" includes-</p>
<p>gunmpletions, maintenance, repairs, and asset replacement activities.</p>
<p>Quality</p>
<p>*/ Supie (England) Regularions 20164), the Water Supply Nu Qual</p>
<p>Low crane Ixcince tacans the looser of a seucrare Ixcice under sochon 178% of tho</p>
<p>Water Industry Act 1991(e):</p>
<p>we crape services" has the meaning givee in sction 2191) of the Water Industry Actl</p>
<p>48/ We industy ice 19 lar the mating given in seties 17ACT) and 2I9C) of the nak managemcat on bchall of</p>
<p>pay one nvironment Agency, or)</p>
<p>tal tood dinory to three.</p>
<p>Cle the purposes of sub-paragraph l wyneed and "coastal crosion" have the meanings given in section I of the Plood and</p>
<p>Water Management Act 20100h);</p>
<p>pg/w9, management" has the meaning gives in section 3 of that Acci.</p>
<p>229,n29 1) Workers engaged in essential or emergency works- we we stricity interccenector,</p>
<p>regree trict heat network as delined in regulation 2 of the Heat Network (Meicnng and</p>
<p>Billing) Regulations 2014(j).</p>
<p>Billing) Regulations 2014.</p>
<p>put mao ommissicaling, maintenance and repair of industrial machinery for use on al</p>
<p>Рд (1 4/3130. There as Do trialt areading envie:</p>
<p>I canations of the saddle point are thie to lowing)</p>
<p>(11))</p>
<p>Mll9. C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on the support of p. We can prove that these equations lead to a singular support for p. We</p>
<p>do this by contradiction.</p>
<p>woRd then that</p>
<p>397199e first two terms obviously are cancelled. Notice that</p>
<p>3'iMat away from 2' - O we have that</p>
<p>91a + 03) - Pª (log(=*) + log(=*)) - 0|</p>
<p>peese the function is harmonic: a sum of i hodonsorplise plus an antiliolomorphic plece:</p>
<p>RI, is is casy to show that (F-) log(a - 6) o d°(a - 6). With these, we would find that p = 0. This argument contradicts that fp = 1, so the assumption that p is smooth does</p>
<p>not work. This argument has boen given before in (5, 23), and we are presenting it bere for completeness</p>
<p>siddhat this indicates is that the demity distribution of narticks le sinoular. Since tot</p>
<p>9 = 0 the original probability distribution is spherically symmetric, it is natural to assume that the associated distribution is spherically summetric. The radius was computed in 118</p>
<p>Pitchas also been chocked macrically in 6. 7. A geocralization to all Calabi- Yau conc conforinal field tboory singularities for this result can be found in (23), where it was seen that</p>
<p>this procedure matches vurious otber supergravity calculations in the abeence of symmetry.</p>
<p>anger q &gt; 0 but very small, we expect that the distribution does not change much, except in the region :' ≥ 0. Thus we expoct that the distribution gives a five dimensional manifold of</p>
<p>R°, possibly with a boundary, and that the region bear 2' = O has no density of eigenvalues present. After all, the effoctive potential repels all particles from the locus :' = 0. This</p>
<p>smolDocling</p>
<p>Y/I"Woekers engaged in essential or emergensy works.</p>
<p>K/""ential or emergency works" inclades</p>
<p>Greedons, maintenance, repairs, and asset replacement activities,</p>
<p>2016(a).</p>
<p>Repain 2 a. he lies Super it Quie)</p>
<p>Supply (Water</p>
<p>Quality</p>
<p>Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the</p>
<p>Qwen2.5-VL</p>
<p>21. (1) Workers encaned in cascatial or emergency works—</p>
<p>"O) "emal or emergency works" includes-</p>
<p>0) mspoctos, mamienance, repam, and asset replacement actioes and analysis of walcr</p>
<p>Watch</p>
<p>Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the</p>
<p>2016(a).</p>
<p>Supply (Wa</p>
<p>Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. "Wille C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on tbe suppost of p. We can prove that these equations lead to a singular support for p. We do this by contradiction.</p>
<p>do this by contradiction.</p>
<p>DEVOTION TO SERVICE</p>
<p>Net incorre heinrat let tormente</p>
<p>Total astets</p>
<p>Working capital</p>
<p>Shareholders equity</p>
<p>Common shares outstanding</p>
<p>$1171</p>
<p>$ 875.228</p>
<p>$ 265,375</p>
<p>$ 436,616</p>
<p>15.971</p>
<p>38,104</p>
<p>$ 1.197,188</p>
<p>$ 295,648</p>
<p>$ 451.169</p>
<p>36.520</p>
<p>Giving back to the and where our</p>
<p>employees live, work, and care for their</p>
<p>families is a serious and dedicated</p>
<p>commitment.</p>
<p>FINANCIAL HIGHLIGHTS</p>
<p>VISA's ongoing commitment to investing in core products and focus on managing costs helped generate a strong finish in 2011.</p>
<p>CROCURR TUAL SALES</p>
<p>BY CORE PRODUCT GROUP</p>
<p>© Fund Gin A Fume Detectior</p>
<p>· Poutis Car Outection</p>
<p>@ Other hodets</p>
<p>60152</p>
<p>1.91</p>
<p>S1015092</p>
<p>$ 287,079</p>
<p>$ 433,666</p>
<p>36,493</p>
<p>DEVOTION TO SERVICE</p>
<p>"Giving back to the and where our</p>
<p>-employees live, work, and care for their</p>
<p>families is a serious</p>
<p>Giving back to the and where our</p>
<p>employees live, work, and care for their</p>
<p>families is a serious</p>
<p>Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures.</p>
<p>340.9</p>
<p>DEVOTION TO SERVICE</p>
<p>22%</p>
</div>
</body>
</html>

pages_json:
[{"page_number": 1, "width": 612.0, "height": 792.0, "elements": [{"type": "section_header", "bbox": [75.333, 90.88667499999997, 536.6712892000002, 134.60335780000003], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion", "level": 1}, {"type": "text", "bbox": [96.914, 186.28416659999993, 517.5758744, 215.79938820000007], "content": "Ahmed Nassar 1 , Andres Marafioti 2 , Matteo Omenetti 1 , Maksym Lysak 1 , Nikolaos Livathinos 1 , Christoph Auer 1 , Lucas Morin 1 , Rafael Teixeira de Lima 1 , Yusik Kim 1 , A. Said Gurbuz 1 , Michele Dolfi 1 , Miquel Farr\u00e9 2 , Peter W. J. Staar 1", "level": 1}, {"type": "text", "bbox": [245.00899999999982, 223.87747500000012, 366.49281699999983, 237.8993882000001], "content": "IBM Research 1 , HuggingFace 2", "level": 1}, {"type": "text", "bbox": [153.43199999999982, 251.5261008, 458.5675583999997, 271.39638820000005], "content": "https://huggingface.co/ds4sd/SmolDocling-256M-preview Version 1.0, March 14 th 2025", "level": 1}, {"type": "section_header", "bbox": [278.3899999999999, 315.2505632, 333.6110687999999, 320.797776], "content": "Abstract", "level": 1}, {"type": "text", "bbox": [107.387, 325.918075, 505.8226149056, 481.5043882], "content": "Weintroduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms - significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon.", "level": 1}, {"type": "section_header", "bbox": [72.0, 501.1055632, 159.21318400000004, 506.652776], "content": "1 Introduction", "level": 1}, {"type": "text", "bbox": [72.0, 517.858075, 541.0973397709998, 629.8083882], "content": "For decades, converting complex digital documents into a structured, machine-processable format has been a significant technical challenge. This challenge primarily stems from the substantial variability in document layouts and styles, as well as the inherently opaque nature of the widely used PDF format, which is optimized for printing rather than semantic parsing. Intricate layout styles and visually challenging elements such as forms, tables, and complex charts can significantly impact the reading order and general understanding of documents. These problems have driven extensive research and development across multiple domains of computer science. On one hand, sophisticated ensemble systems emerged, which decompose the conversion problem into several sub-tasks (e.g. OCR, layout analysis, table structure recognition, classification) and tackle each sub-task independently. Although such systems can achieve high-quality results for many document types while maintaining relatively low computational demands, they are often difficult to tune and generalize.", "level": 1}, {"type": "text", "bbox": [72.0, 632.428075, 541.6859401416003, 722.5603882], "content": "On the other hand, in the recent past, great interest has developed around large foundational multimodal models that can solve the whole conversion task in one shot, while simultaneously offering flexible querying and parametrization through prompts. This approach has been made possible by the advent of multimodal pre-training of large visionlanguage models (LVLMs), which opened up a vast array of opportunities for leveraging diverse data sources, including PDF documents. However, the literature on this topic highlights a significant gap in the availability of high-quality and open-access datasets suitable for training robust multi-modal models for the task of document understanding. Furthermore, relying on LVLMs may introduce common issues associated with such models, including hallucinations and the use of significant computational resources, making them impractical from both a quality and cost perspective.", "level": 1}]}, {"page_number": 2, "width": 612.0, "height": 792.0, "elements": [{"type": "picture", "bbox": [98.08104705810547, 71.2328872680664, 512.7471923828125, 213.59715270996094], "content": "", "level": 1}, {"type": "text", "bbox": [72.0, 233.74307499999998, 541.538533512, 280.2383882], "content": "Figure 1: SmolDocling/SmolVLM architecture. SmolDocling converts images of document pages to DocTags sequences. First, input images are encoded using a vision encoder and reshaped via projection and pooling. Then, the projected embeddings are concatenated with the text embeddings of the user prompt, possibly with interleaving. Finally, the sequence is used by an LLM to autoregressively predict the DocTags sequence.", "level": 2}, {"type": "text", "bbox": [72.0, 298.831075, 541.5339365599999, 334.4173882], "content": "In this work, we outline how we close the gaps left by publicly available datasets and establish a training approach to achieve end-to-end, full-featured document conversion through a vision-language model, which we name SmolDocling . Our main contributions can be summarized as follows:", "level": 1}, {"type": "list", "bbox": [99.387, 342.095075, 540.0046143088001, 410.40938819999997], "content": "SmolDocling: An ultra-compact VLM for end-to-end document conversion, based on Hugging Face's most recent SmolVLM-256M [56], which is between 5 and 10 times smaller in parameters than comparable VLMs tuned on document understanding tasks. Hence, we reduce the computational complexity to roughly the same order of magnitude as typical representatives of ensemble methods. Uniquely, SmolDocling learns and produces a unified document representation that captures content, structure and spatial location of all elements in a page.", "level": 2}, {"type": "list", "bbox": [99.387, 411.636075, 541.0976111547999, 469.0413882], "content": "We augment existing document pre-training datasets with additional feature annotations, and create new datasets for tasks with insufficient coverage in open multi-modal training datasets, including code listings, equations, and chart understanding. Importantly, we construct full-page ground truth data that combines key annotation features such as layout, table structure, code listings, charts, and formulas in their natural context, and derive instruction datasets from all training datasets used.", "level": 2}, {"type": "list", "bbox": [99.387, 470.269075, 540.0018592584, 494.9463882], "content": "We introduce DocTags , a document markup format optimized to efficiently represent the full content and layout characteristics of a document, inspired by [55].", "level": 2}, {"type": "list", "bbox": [99.387, 496.174075, 539.9973633439998, 520.8513882], "content": "We evaluate the achieved prediction quality of SmolDocling on several tasks, including comparisons to popular community models.", "level": 2}, {"type": "section_header", "bbox": [72.0, 540.8875631999999, 164.712576, 546.434776], "content": "2 Related Work", "level": 1}, {"type": "section_header", "bbox": [72.0, 564.6671666, 273.66294919999996, 569.289813], "content": "2.1 Large Vision-Language Models (LVLMs)", "level": 1}, {"type": "text", "bbox": [71.522, 577.883075, 541.6807994399998, 722.5603882], "content": "With the advent of large language models (LLMs) [74, 66, 82, 32, 90, 4, 8, 25, 2] characterized by their powerful contextual understanding and adaptability, several approaches have emerged to extend them into large visionlanguage models. Some proprietary, closed-source LLMs have been adapted into multimodal large language models (MLLMs), such as GPT-4o [65], Gemini [80], and Claude 3.5 [5], demonstrating exceptional capabilities across various modalities, including vision. Open-source methods are actively advancing to match the capabilities of proprietary MLLMs. BLIP-2 [47] is one of the earliest works combining a vision encoder with a frozen LLM (OPT [102] or FlanT5 [14]) using a lightweight transformer (Q-former [101]). Building upon BLIP-2, MiniGPT-4 [105] integrated a frozen visual encoder with a frozen LLM (Vicuna [85]) using a Q-Former network and a single projection layer. LLaVA [52, 51] employs a similar architecture but uses a minimal adapter layer. LLaVA-OneVision [45] and LLaVA-NeXTInterleave [46] build upon LLaVA to support multi-image, higher resolution, and video understanding. For handling high resolutions and text-image compositions and comprehension, InternLM-XComposer introduces a family of models [99, 100, 20, 19] designed specifically for these tasks. Qwen-VL [8] introduces a position-aware adapter to address efficiency issues caused by the vision encoder generating long image feature sequences. Qwen2.5-VL [9]", "level": 1}]}, {"page_number": 3, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 70.75507499999992, 540.2538960923998, 95.43238819999999], "content": "employs windowed attention with 2D rotary positional embeddings to process native-resolution inputs efficiently and integrates an vision-language merger for dynamic feature compression.", "level": 1}, {"type": "text", "bbox": [71.751, 98.05207499999995, 541.0914120239996, 155.45738820000008], "content": "In this work, we build upon the SmolVLM architecture approach [56]. Precisely, we adopt SmolVLM-256M, a compact variant of base SmolVLM-2.2B model, inspired by the architecture of Idefics3 [41] which employs the shape-optimized SigLIP encoder as its visual backbone. It applies an aggressive pixel shuffle strategy to compress visual features, reduces the number of image hidden states and introduces new special tokens to separate sub-images and improve tokenization efficiency.", "level": 1}, {"type": "section_header", "bbox": [72.0, 178.95116659999997, 294.42500759999996, 183.57381299999997], "content": "2.2 State of the Art in Document Understanding", "level": 1}, {"type": "text", "bbox": [72.0, 194.742075, 540.0030975999999, 263.0563882], "content": "Several powerful document understanding solutions have emerged on the market in the past decade as commercial cloud offerings on hyperscalers [24, 62, 3, 6], frontier models such as GPT-4o [65] or Claude [5], or as open-source libraries such as Docling [53], Grobid [1], Marker [67], MinerU [86] or Unstructured [84]. Typical tasks that document understanding encompasses are document classification [96, 29], OCR [48], layout analysis [95, 70], table recognition [103, 55, 78], key-value extraction [29, 35, 31], chart understanding [49, 57, 50, 26, 60, 93, 94, 13, 58, 33, 34], equation recognition [77, 68, 18, 97, 23], and more.", "level": 1}, {"type": "text", "bbox": [71.631, 265.67607499999997, 541.6807994399999, 323.08038819999996], "content": "Ensemble systems, such as the aforementioned open-source libraries [53, 1, 67, 86], implement a pipeline in source code, which conditionally applies specialized, single-task models on a given input document, composing the prediction results into a meaningful document representation. Typically, each task entails human-crafted handling of preprocessing and post-processing logic (i.e. setting confidence thresholds for layout detections, matching layout elements to text cells, conditionally applying models).", "level": 1}, {"type": "text", "bbox": [72.0, 325.701075, 541.6814887280002, 437.6513882], "content": "Conversely, multi-task models aim at providing a single model capable of handling various document-understanding related tasks simultaneously, benefiting from shared context and shared representations learned across these different tasks. OCR-reliant methods, such as LayoutLM [96, 95, 29] and UDOP [79], use text extracted from an external OCR engine, along with image and text bounding box locations, as input. In contrast, OCR-free methods such as Donut [35], Dessurt [17], DocParser [76], Pix2Struct [43] are transformer-based models trained end-to-end to take an image as input and directly output text. Large Vision Language Models (LVLMs) such as LLaVA [51], LLaVAOneVision [45], UReader [98], Kosmos-2 [69], and Qwen-VL [8] all leverage various vision encoders, projection adapters, and LLMs. While they may use different image patching techniques and instruction-tuning datasets to handle a range of vision tasks, they commonly evaluate their document understanding capabilities on datasets like DocVQA [59] and mPLUG-DocOwl 1.5 datasets [27], primarily focusing on question answering and reasoning.", "level": 1}, {"type": "text", "bbox": [72.0, 440.271075, 541.0914120239995, 584.9483882], "content": "Our work focuses primarily on document understanding tasks related to conversion and structural recognition, aiming to represent documents with high accuracy and completeness. The closest works to our approach are Nougat [12], DocOwl 2 [28], GOT [89], and Qwen2.5-VL [9]. DocOwl 2 and GOT are dedicated to precise conversion and recognition of document structures. On one hand, DocOwl 2 employs a dynamic shape-adaptive cropping module to process high-resolution images, feeding them into a ViT vision encoder. A dedicated module compresses the processed visual features before feeding them into a LLaMa-based LLM. On the other hand, GOT focuses on converting diverse elements-such as text, formulas, molecular diagrams, tables, charts, sheet music, and geometric shapes-into structured formats. It achieves a lightweight architecture by utilizing a SAM vision encoder [37], a linear projection layer, and a Qwen-0.5B LLM. Building on these approaches, our work not only extends the range of tasks but also broadens the types of documents handled, moving beyond a strictly scientific focus to support a more diverse array of document formats. In parallel, Qwen2.5-VL introduces the Omni-Parsing strategy, that integrates multiple document elements into a unified HTML-based representation. This format encodes layout bounding box information and reading order for various document elements such as paragraphs and charts.", "level": 1}, {"type": "section_header", "bbox": [72.0, 611.4215632, 159.64357120000003, 616.9687759999999], "content": "3 SmolDocling", "level": 1}, {"type": "text", "bbox": [71.701, 632.428075, 541.1003017568001, 722.5603882], "content": "Building on Hugging Face's SmolVLM [56], SmolDocling converts document page images into sequences representing their content and structure. It produces a unified page representation (DocTags markup) that is optimized for the LLM backbone and interpretable by Docling [53]. This new DocTags format covers the reproduction of captions, charts, forms, code, equations, tables, footnotes, lists, page footers and headers, section headings, and text. SmolDocling both recognizes the type and location of these elements as well as performs OCR within them. Our approach enables full-page conversion with reading-order and hierarchical linking between elements (e.g., linking captions with figures and tables), providing bounding boxes for each component, supporting isolated predictions on individual elements (e.g., cropped tables) and grouping for lists and nested lists.", "level": 1}]}, {"page_number": 4, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [36.100292821220854, 18.980167062505075, 163.5130835188953, 23.341463272764145], "content": "Fig. 3. OTSL description of table structure: A - table example; B - graphical repre- sentation of table structure; C - mapping structure on a grid; D - OTSL structure", "level": 1}, {"type": "text", "bbox": [36.10029229918238, 23.308976990619044, 164.2209318340661, 27.637787011495902], "content": "Fig. 3. OTSL description of table structure: A - table example; B - graphical repre- sentation of table structure; C - mapping structure on a grid; D - OTSL structure", "level": 1}, {"type": "text", "bbox": [123.87354733573558, 48.09756119613826, 145.10901118131864, 53.04878043231702], "content": "ucccc c c", "level": 1}, {"type": "text", "bbox": [128.12063964106105, 52.278705620740766, 136.61482568757268, 56.607515641617624], "content": "ucccc c c", "level": 1}, {"type": "text", "bbox": [135.9069765908188, 52.27870570326104, 145.10901147453976, 56.6075157241379], "content": "c c", "level": 1}, {"type": "text", "bbox": [109.71657024180797, 66.48780495487813, 141.56976537483456, 68.60975605609758], "content": "2 - horizonta merces. C.", "level": 1}, {"type": "text", "bbox": [110.42441812095446, 69.26096036270587, 139.44622044653588, 72.92379961114011], "content": "I - vertical merges: \"C\", U", "level": 1}, {"type": "text", "bbox": [54.50436035893895, 70.59290190799766, 62.290697568241285, 75.68292660873976], "content": "\u00b7 c", "level": 1}, {"type": "text", "bbox": [26.89825347805241, 86.90918586552834, 167.05232324549425, 90.57202511396258], "content": "Table 2. TSR and cell detection results compared between OTSL and HTML on the PubTabNet (22], FinTabNet [21] and PubTables-IM [14] data sets using Table-", "level": 1}, {"type": "text", "bbox": [26.89825755703121, 91.23799575441467, 166.34447848726379, 95.56680577529153], "content": "Table 2. TSR and cell detection results compared between OTSL and HTML on the PubTabNet (22], FinTabNet [21] and PubTables-IM [14] data sets using Table-", "level": 1}, {"type": "text", "bbox": [26.89825668106829, 95.48780497865857, 96.26744272757992, 99.89561589165692], "content": "Former [9) (with enc=6, dec=6, heads=8).", "level": 1}, {"type": "text", "bbox": [37.51598861984011, 107.51219518516268, 52.38081420123546, 111.04878035386173], "content": "Data set", "level": 1}, {"type": "text", "bbox": [58.75145322882752, 106.80487809634155, 75.73982532185077, 111.04878029878046], "content": "Language", "level": 1}, {"type": "text", "bbox": [34.684593965338316, 117.41463433536592, 54.50436013650276, 121.65853653780482], "content": "PubTabNet", "level": 1}, {"type": "text", "bbox": [35.39244155373063, 128.02439037784552, 53.79651132117249, 131.56097554654468], "content": "FinTabNet", "level": 1}, {"type": "text", "bbox": [32.56104692453972, 138.63414640467477, 57.3357562268653, 142.17073157337393], "content": "Pub'Tables-1M", "level": 1}, {"type": "text", "bbox": [62.29069786317828, 115.29268301361799, 73.61627925852713, 118.82926818231704], "content": "OTSL", "level": 1}, {"type": "text", "bbox": [61.582848686791436, 120.95121963729684, 73.61627891934958, 124.48780480599589], "content": "HTML", "level": 1}, {"type": "text", "bbox": [62.29069780714025, 125.90243909644312, 72.90843036527978, 129.43902426514228], "content": "OTSL", "level": 1}, {"type": "text", "bbox": [61.58284866024709, 130.8536585841464, 73.61627889280523, 135.0975607865853], "content": "HTML", "level": 1}, {"type": "text", "bbox": [62.29069780714025, 136.5121951940041, 72.90843036527978, 140.04878036270327], "content": "OTSL", "level": 1}, {"type": "text", "bbox": [61.582849037766465, 141.46341472093502, 73.6162792703246, 144.99999988963418], "content": "HTMLI", "level": 1}, {"type": "text", "bbox": [26.89825579625727, 156.3170733135163, 35.392441842768896, 159.85365848221545], "content": "5.3", "level": 1}, {"type": "text", "bbox": [25.48255898895346, 156.17014605526674, 76.44767526802322, 162.8298537796926], "content": "15.3 Qualitative Results", "level": 1}, {"type": "text", "bbox": [23.35901285956393, 175.41463444764236, 146.52471053398253, 186.80480158840396], "content": "The OTS: vocabulary is comprised of the following tokens.", "level": 1}, {"type": "text", "bbox": [28.313955196057375, 186.02439035538623, 164.92878077745272, 191.13361169336758], "content": "\"C\" cell - a new table cell that either has or does not have cell content", "level": 1}, {"type": "text", "bbox": [28.313951069888656, 191.6829269790651, 173.42296269779567, 196.7943632953303], "content": "\"L\" cell - left-looking cell, merging with the left neighbor cell to create a", "level": 1}, {"type": "text", "bbox": [28.313951069888656, 202.29268307662608, 173.42296269779567, 207.24390231280483], "content": "\"U\" cell - up-looking cell, merging with the upper neighbor cell to create a span", "level": 1}, {"type": "text", "bbox": [28.31395336449855, 209.36585378363827, 38.22383708542878, 212.90243895233743], "content": "\"U\" cell - up-looking cell, merging with the upper neighbor cell to create a span", "level": 1}, {"type": "text", "bbox": [29.021803656337184, 213.44363251136008, 162.09738505168605, 218.56097559516184], "content": "\"X\" cell - cross cell, to merge with both left and upper neighbor cells", "level": 1}, {"type": "text", "bbox": [29.021803298873518, 219.10438407297715, 106.88517539189678, 223.5121949859755], "content": "\"NL\" - new-line, switch to the next row.", "level": 1}, {"type": "text", "bbox": [28.313955423158863, 229.09394569644826, 172.71511821385653, 234.12195116402438], "content": "A notable attribute of OTSL is that it has the capability of achieving lossless", "level": 1}, {"type": "text", "bbox": [15.572674489389534, 239.07317087378055, 22.651162861482558, 245.43902417743902], "content": "D", "level": 1}, {"type": "text", "bbox": [23.359013775048368, 241.0814195648893, 152.1875021471414, 246.14634133719505], "content": "expression using a data frame called dif in R where you want to summarize x by month:....", "level": 1}, {"type": "text", "bbox": [17.69622050552327, 247.56097574369926, 43.17877864505815, 251.0975609123983], "content": "require(plyr)", "level": 1}, {"type": "text", "bbox": [18.404069224757773, 251.7369520232412, 50.965115736385684, 255.39979127167544], "content": "df <- data. frame(|", "level": 1}, {"type": "text", "bbox": [22.651163510343974, 256.0487805745936, 65.8299425801114, 259.7286013232034], "content": "x = runif(120, 1, 168),", "level": 1}, {"type": "text", "bbox": [22.651163392369167, 263.72442592908055, 70.78488432260173, 268.07317059573165], "content": "2 = runif(120, 1.7, 20.7), y = runif(120, 7, 334).", "level": 1}, {"type": "text", "bbox": [22.651164273748336, 259.5853659012196, 65.12209196491445, 263.8292681036585], "content": "2 = runif(120, 1.7, 20.7), y = runif(120, 7, 334).", "level": 1}, {"type": "text", "bbox": [22.651161964874053, 268.0532359338207, 72.2005805695252, 272.38204595469756], "content": "month = rep(c(5,6,7,8),30), week = sample(1:4, 120, TRUE)", "level": 1}, {"type": "text", "bbox": [22.651163454305934, 272.3170732536586, 75.73982624500361, 276.7108559708596], "content": "month = rep(c(5,6,7,8),30), week = sample(1:4, 120, TRUE)", "level": 1}, {"type": "text", "bbox": [19.111919713614295, 285.03549055403784, 85.64971041128871, 289.3643005749147], "content": "ddply(df, \u00b7 (month, week), sumarize, sd = round(sd(x), 2))", "level": 1}, {"type": "text", "bbox": [29.729653094145604, 292.6941545612348, 69.3691854364745, 297.78048763150406], "content": "ddply(df, \u00b7 (month, week), sumarize, sd = round(sd(x), 2))", "level": 1}, {"type": "text", "bbox": [194.65842814440416, 15.560975775406519, 319.94767233045064, 21.926829079064987], "content": "\u2039picture><loc_102><loc_85><loc_416><loc_160>~", "level": 1}, {"type": "text", "bbox": [196.01325515218164, 22.233914747872973, 230.8195935602695, 28.69291463650609], "content": "<flow_chart>~", "level": 1}, {"type": "text", "bbox": [194.65843009806682, 50.219512223862466, 227.92732544690406, 55.17073177760085], "content": "\u2039/picture>~", "level": 1}, {"type": "text", "bbox": [194.65843165710388, 62.95121962774397, 308.62209444780154, 69.31707293140244], "content": "<otsl><loc_63><loc_78><loc_436><loc_158>*|", "level": 1}, {"type": "text", "bbox": [194.6584321083575, 69.92693126283075, 325.61046699207844, 75.68292657723578], "content": "\u2039ched\u203aData set. <ched>Language<ched>TEDs. <Icel>.", "level": 1}, {"type": "picture", "bbox": [71.95040130615234, 71.29930877685547, 538.4942016601562, 371.3829040527344], "content": "", "level": 1}, {"type": "text", "bbox": [29.729651941424393, 289.29268301361793, 76.44767519723835, 292.82926818231704], "content": "mean = round(mean(x), 2),", "level": 1}, {"type": "text", "bbox": [72.0, 378.186075, 541.0909928319995, 435.59038819999995], "content": "Figure 2: DocTags format describes key features of document elements: type of an element (text, picture, table, code, etc.), location on the page, and content. Nested tags convey additional information: pictures and tables may nest captions, table structure is represented in OTSL tags, lists nest list items, code and pictures carry a classification. All DocTags output is taken from SmolDocling predictions, artificial line-breaks and dots (...) were inserted for better readability. A, B, C - snippets of various pages from [55], D - from [92]", "level": 1}, {"type": "section_header", "bbox": [72.0, 460.7301666, 182.44538359999999, 465.35281299999997], "content": "3.1 Model Architecture", "level": 1}, {"type": "text", "bbox": [71.701, 473.918075, 540.0031329760001, 553.1403882], "content": "The model architecture is illustrated in Figure 1. SmolVLM-256M relies on SigLIP base patch-16/512 (93M) as visual backbone and compared to the 2.2B version of the same model, its training data was rebalanced to emphasize document understanding (41%) and image captioning (14%), combining The Cauldron [42], Docmatix [41] datasets with the addition of MathWriting [23]. It uses lightweight variant of the SmolLM-2 family (135M) [2] as the language backbone and employs a radical pixel shuffle method that compresses each 512x512 image patch into 64 visual tokens. Last but not least, tokenization efficiency was also improved in SmolVLM-256M by increasing the pixel-to-token ratio to 4096 pixels per token and introducing special tokens for sub-image separators.", "level": 1}, {"type": "section_header", "bbox": [72.0, 570.1251666000001, 132.54272019999996, 574.747813], "content": "3.2 DocTags", "level": 1}, {"type": "text", "bbox": [71.701, 583.313075, 541.5381234279996, 684.3533882], "content": "To train VLMs for standardized document conversion, we introduce the DocTags markup format, illustrated in Figure 3. Inspired by OTSL [55], DocTags define a structured vocabulary of unambiguous tags and rules that explicitly separate textual content from document structure. This disentanglement improves performances of Image-to-Sequence models by minimizing confusions. In contrast, direct conversion to standard formats such as HTML or Markdown is often lossy and ambiguous, lacking explicit structural markup, failing to preserve page layout context, and increasing total tokens count. Pieces of text content that represent basic elements in DocTags are enclosed in XML-style tags. Tags define document block types such as: text, caption, footnote, formula, title, list_item, page_footer, page_header, picture, section_header, otsl, code, document_index, etc. Each element can nest additional location tags encoding its position on the page as a bounding box, represented in DocTags as <loc_ x1 ><loc_ y1 ><loc_ x2 ><loc_ y2 > .", "level": 1}, {"type": "text", "bbox": [72.0, 686.974075, 540.0040938599997, 722.5603882], "content": "Special blocks, such as tables and images, nest additional descriptors for captions, tabular structure, or image classes. To encode table structures, we fully include the OTSL vocabulary into DocTags . OTSL has the advantage of being minimalist and supporting cell spans and header information. Its tokens are interleaved with text, allowing", "level": 1}, {"type": "text", "bbox": [367.37354470661336, 70.02439044085372, 475.6744167996367, 74.97560967703248], "content": ",<ched>mAP(0.75)<ched>Inference..<nl>~", "level": 1}]}, {"page_number": 5, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 32.2990749999999, 500.8301544, 46.06738819999998], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion simultaneous encoding of both table structure and content. In order to promote robust visual-semantic alignment in our document understanding pipeline, we maintain a uniform DocTags representation for cropped page elements (e.g., tables, code, equations) that is identical to their full-page counterparts. This consistency ensures that the model can leverage similar visual cues whether an element is presented in isolation or in the context of an entire page, thereby strengthening the learning signal for both computer vision and downstream machine learning tasks.", "level": 1}, {"type": "text", "bbox": [72.0, 70.75507499999992, 541.0952527039997, 128.15938820000008], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion simultaneous encoding of both table structure and content. In order to promote robust visual-semantic alignment in our document understanding pipeline, we maintain a uniform DocTags representation for cropped page elements (e.g., tables, code, equations) that is identical to their full-page counterparts. This consistency ensures that the model can leverage similar visual cues whether an element is presented in isolation or in the context of an entire page, thereby strengthening the learning signal for both computer vision and downstream machine learning tasks.", "level": 1}, {"type": "text", "bbox": [72.0, 130.780075, 424.48675059999994, 144.5483882000001], "content": "For a complete list of tags and detailed description of the format, consult the appendix.", "level": 1}, {"type": "section_header", "bbox": [72.0, 161.9781666, 164.8215442, 166.60081300000002], "content": "3.3 Model Training", "level": 1}, {"type": "text", "bbox": [71.522, 175.343075, 541.0989836000002, 254.5663882], "content": "We employ a curriculum learning approach to progressively align our model for document conversion, ensuring faster convergence. As an initial step, we incorporate DocTags as tokens in our tokenizer. To align the LLM part we freeze the vision encoder and train only the remaining network to adapt it to the new output format which it hasn't seen before. To ensure comprehensive coverage of all DocTags, we maintain a balanced mix of tasks and data types during training. Next, we unfreeze the vision encoder and train the model on our pretraining datasets (see Sec. 4.1), along with all task-specific conversion datasets, including tables, code, equations, and charts. Finally, we fine-tune using all available datasets.", "level": 1}, {"type": "section_header", "bbox": [72.0, 274.97456320000003, 115.25391359999999, 280.521776], "content": "4 Data", "level": 1}, {"type": "text", "bbox": [72.0, 292.343075, 541.5384139608004, 338.8383882], "content": "For document understanding tasks, each page image represents the visual layout of textual elements-such as paragraphs, formulas, tables, code, or figures with embedded textual content. Publicly available datasets that unify all these characteristics remain limited, and existing corpora often suffer from small sample sizes or fragmented formats. In this section, we explain how we mitigate these issues in both pre-training and task-specific training scenarios.", "level": 1}, {"type": "section_header", "bbox": [72.0, 356.26816660000003, 188.8015224, 360.890813], "content": "4.1 Pre-training datasets", "level": 1}, {"type": "text", "bbox": [71.771, 369.633075, 540.3687153809997, 437.9473882], "content": "In the domain of document understanding, currently available open-access document pre-training datasets include OCR-IDL [10], IIT-CDIP [44], CCpdf [83], and WordScape [88]. OCR-IDL and IIT-CDIP, industry-sourced from the 1990s, provide only page images and OCR text. CCpdf and WordScape, derived from Common Crawl, focus on raw PDFs and Word documents, respectively. While WordScape enriches documents with layout, table structure, and topic information via XML markup, its visual variability is limited. Notably, none of these datasets offer rich annotations for all page elements.", "level": 1}, {"type": "text", "bbox": [71.631, 440.567075, 541.0997608090004, 541.6083882], "content": "DocLayNet-PT. Given the lack of public multimodal document pre-training data with sufficient annotation features, we constructed a new document pre-training dataset named DocLayNet-PT. DocLayNet-PT is a 1.4M pages dataset extracted from the DocFM dataset [25] of unique PDF documents, sourced from CommonCrawl, Wikipedia, and business-related documents. We focused on this dataset to look for documents including visually distinct content such as equations, tables, code and charts with colorful layouts. This dataset was augmented with weak annotations through a series of processing steps. First, we applied PDF parsing to all pages to obtain the text layout including positional information using Docling [7, 53]. Next, we performed an enrichment step to obtain an enhanced, machine-friendly representation of each page. Specifically, we provide annotations for layout elements, table structure, language, topic, and figure classification on each page in DocTag format.", "level": 1}, {"type": "text", "bbox": [72.0, 544.229075, 540.2623427655999, 590.7243882], "content": "Docmatix. To preserve SmolVLM's original DocVQA capabilities, we adapted the same weak annotation strategy used for DocLayNet-PT and applied it to the 1.3M documents inside Docmatix dataset [41]. In addition to the 9.5 million DocVQA questions, we introduced another instruction requiring the full conversion of multi-page documents to DocTags.", "level": 1}, {"type": "section_header", "bbox": [72.0, 608.1541665999999, 193.74297199999998, 612.776813], "content": "4.2 Task-specific Datasets", "level": 1}, {"type": "text", "bbox": [71.522, 621.519075, 541.5291686679997, 722.5603882], "content": "Layout. To optimize prediction quality for document layout and table structure, we sampled 76K pages from DocLayNet-PT. Each page underwent human annotation and rigorous quality review, resulting in a high-quality ground-truth dataset. Of these, 60K pages were utilized for fine-tuning, constituting what we refer to as DocLayNet v2. Additionally, we extracted 63K pages from WordScape, specifically filtering for pages containing both text and tables, and utilized their inherently structure-preserving format as a reliable source of ground truth. To further enhance the model's adaptability to diverse layouts, colors, and fonts, we synthetically generated 250K pages using content from Wikipedia, creating the dataset known as SynthDocNet. This synthetic dataset enables the controlled generation of various document types with precise annotations. All datasets share a unified annotation format and class definitions, ensuring consistency and comprehensive document representation.", "level": 1}]}, {"page_number": 6, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [71.631, 70.75507499999992, 541.0960268544001, 128.15938820000008], "content": "Tables. For the table structure recognition task we trained our model on public datasets such as PubTables-1M [78], FinTabNet [103], WikiTableSet [54] and tabular information extracted from WordScape documents. We transformed table structure information from the original ground-truth to OTSL format [55]. Then, we interleaved each cell tag with ground-truth text. This yields a condensed sequence which represents the structure of the table (including merged cells, and header declaration) along with its textual content.", "level": 1}, {"type": "text", "bbox": [72.0, 130.780075, 541.0935758243999, 220.91138820000003], "content": "Charts. For chart reconstruction, several datasets are publicly available [58, 33, 34, 93] that consist of both synthetic and web-scraped charts. However, these datasets are limited by either the quantity of real-world data or the visual diversity of synthetic examples. This has prompted recent studies [50, 26, 60, 94, 93, 13, 57] to crawl or render additional datasets, which unfortunately are rarely shared publicly. To address these gaps, we generated our own chart dataset comprising four types of charts: line, pie, bar, and stacked bar. Using data from 90,000 tables sourced from the FinTabNet dataset [103], we generated a comprehensive dataset containing 5,000 line charts, 380,000 pie charts, 380,000 bar charts, and 77,000 stacked bar charts. Each data point was rendered using three different visualization libraries (details provided in the Appendix), yielding a total of 2.5 million visually diverse charts.", "level": 1}, {"type": "text", "bbox": [71.631, 223.53207499999996, 541.6875238239999, 313.6633882], "content": "Code. Technical books and scientific documents frequently include code snippets, making it essential for documentprocessing models to accurately interpret code structure, particularly indentation, which carries semantic significance in certain programming languages. Several publicly available datasets provide extensive collections of code snippets with permissive licenses in text format [38, 71], typically aggregated from sources like GitHub. However, a common limitation is that these datasets contain only text-based code snippets, lacking any image-based representation. To address this gap, we leveraged LaTeX and Pygments [16] to generate visually diverse renderings of code (details provided in the Appendix). Our dataset includes 9.3M million rendered code snippets generated at 120 dpi, covering 56 programming languages.", "level": 1}, {"type": "text", "bbox": [72.0, 316.28407500000003, 541.5355945449999, 406.4153882], "content": "Equations. Accurately parsing mathematical formulas is crucial for document-processing models, as formulas often contain nuanced structures and semantics critical for scientific understanding. To effectively train our model for formula transcription, we combined publicly available datasets with a custom-rendered dataset. The public datasets [18, 22, 11, 23] collectively comprise approximately 730k unique formulas. In addition, we extracted 4.7 million formulas from arXiv by applying a regular expression to the source LaTeX code. The extracted formulas underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized LaTex code. Subsequently, these normalized equations were visually rendered using LaTeX (details provided in the Appendix). Overall, our dataset includes 5.5 million unique formulas, each rendered at a resolution of 120 dpi.", "level": 1}, {"type": "table", "bbox": [188.82147216796875, 423.91522216796875, 422.5161437988281, 670.28271484375], "content": "| Dataset Name                     | Size   | Type       |\n|----------------------------------|--------|------------|\n| TheCauldron [42]                 | 2.63M  |            |\n| DocLayNet-PT-Instruct            | 1.62M  | Multi Task |\n| DocMatix w/ Doc Conversion* [41] | 952K   |            |\n| SynthChartNet*                   | 2.5M   |            |\n| PlotQA [61]                      | 225K   |            |\n| FigureQA [33]                    | 100K   | Chart      |\n| Unichart [57]                    | 600K   |            |\n| ChartQA [58]                     | 28K    |            |\n| DocLayNet-PT                     | 1.40M  |            |\n| Doc4MLlaVa - RVLCDIP             | 742K   |            |\n| SynthDocNet                      | 375K   | Conversion |\n| DocLayNet [70]                   | 81K    |            |\n| DocLayNet v2                     | 76K    |            |\n| WordScape-PT                     | 63K    |            |\n| PubTable1M [78]                  | 1M     |            |\n| WordScape Tables [88]            | 207K   | Table      |\n| WikiTableSet (EN) [54]           | 204K   |            |\n| FinTabNet [103]                  | 90K    |            |\n| SynthFormulaNet*                 | 5.5M   | Formula    |\n| SynthCodeNet*                    | 9.3M   | Code       |", "level": 1}, {"type": "text", "bbox": [71.701, 675.237075, 539.99904636, 699.9143882], "content": "Table 1: Datasets. Overview of the datasets used for the training and evaluation of SmolDocling. Datasets we contribute as part of this publication are denoted with (*).", "level": 2}]}, {"page_number": 7, "width": 612.0, "height": 792.0, "elements": [{"type": "section_header", "bbox": [72.0, 77.39016659999993, 268.7713126, 82.01281299999994], "content": "4.3 Document Instruction Tuning datasets", "level": 1}, {"type": "text", "bbox": [71.631, 90.56207499999994, 541.0934817959995, 158.87638820000006], "content": "To reinforce the recognition of different page elements and introduce document-related features and no-code pipelines, we leveraged rule-based techniques and the Granite-3.1-2b-instruct [25] LLM. Using samples from DocLayNet-PT pages, we generated one instruction by randomly sampling layout elements from a page. These instructions included tasks such as 'Perform OCR at bbox,' 'Identify page element type at bbox,' and 'Extract all section headers from the page.' In addition, we train with Cauldron [42] to avoid catastrophic forgetting with the amount of conversation datasets introduced.", "level": 1}, {"type": "table", "bbox": [73.36613464355469, 166.79290771484375, 540.7066040039062, 316.1184997558594], "content": "| Method             | Size   |   Edit Distance \u2193 |   F1-score \u2191 |   Precision \u2191 |   Recall \u2191 |   BLEU \u2191 |   METEOR \u2191 |\n|--------------------|--------|-------------------|--------------|---------------|------------|----------|------------|\n| Full-page          |        |                   |              |               |            |          |            |\n| Qwen2.5 VL [9]     | 7B     |              0.56 |         0.72 |          0.8  |       0.7  |     0.46 |       0.57 |\n| GOT [89]           | 580M   |              0.61 |         0.69 |          0.71 |       0.73 |     0.48 |       0.59 |\n| Nougat (base) [12] | 350M   |              0.62 |         0.66 |          0.72 |       0.67 |     0.44 |       0.54 |\n| SmolDocling (Ours) | 256M   |              0.48 |         0.8  |          0.89 |       0.79 |     0.58 |       0.67 |\n| Code listings      |        |                   |              |               |            |          |            |\n| SmolDocling (Ours) | 256M   |              0.11 |         0.92 |          0.94 |       0.91 |     0.87 |       0.89 |\n| Equations          |        |                   |              |               |            |          |            |\n| Qwen2.5 VL [9]     | 7B     |              0.22 |         0.89 |          0.91 |       0.87 |     0.68 |       0.77 |\n| GOT [89]           | 580M   |              0.11 |         0.95 |          0.95 |       0.96 |     0.85 |       0.91 |\n| Nougat (base) [12] | 350M   |              0.62 |         0.6  |          0.6  |       0.53 |     0.33 |       0.41 |\n| SmolDocling (Ours) | 256M   |              0.11 |         0.95 |          0.96 |       0.95 |     0.83 |       0.89 |", "level": 1}, {"type": "text", "bbox": [71.701, 329.76707500000003, 540.0024400684003, 365.3533882], "content": "Table 2: Structured document text recognition. We evaluate OCR performance and document formatting on DocLayNet, focusing on textual elements, excluding tables. Text accuracy is measured through multiple metrics (Edit Distance, F1-Score, Precision, Recall, BLEU and METEOR) as proposed in [12].", "level": 1}, {"type": "section_header", "bbox": [72.0, 397.4415632, 159.34469120000003, 402.98877600000003], "content": "5 Experiments", "level": 1}, {"type": "text", "bbox": [72.0, 413.91107500000004, 539.9988933828004, 471.3153882], "content": "In this section, we assess the performance of SmolDocling on document understanding tasks, and compare the results to previously published models and baselines. Capturing the qualities and defects of SmolDocling and other models in comparable numbers is a non-trivial effort for reasons we outline further below. Therefore, we report results on standard evaluation tasks and metrics independently (see section 5.2), but also provide a qualitative assessment of output characteristics (see section 5.3).", "level": 1}, {"type": "section_header", "bbox": [72.0, 487.6541666, 202.2211446, 492.27681299999995], "content": "5.1 Implementation Details", "level": 1}, {"type": "text", "bbox": [71.751, 500.82707500000004, 540.1908480214006, 569.1403882], "content": "Our training was conducted using 64 NVIDIA A100 80GB GPUs, with each epoch requiring 38 hours. The model was trained for a total of 4 epochs using the AdamW optimizer with a learning rate of 2 \u00d7 10 -4 and 2 \u00d7 10 -6 for the vision encoder once unfrozen. We also employed gradient clipping to 1.0 for stability and a warmup ratio of 0.03. For inference, using VLLM [39] on an A100 GPU, SmolDocling achieves a page conversion time of 0.35 seconds while occupying only 0.489 GB of VRAM. The model supports a maximum sequence length of 8,192 tokens and is trained to process up to three pages at a time.", "level": 1}, {"type": "section_header", "bbox": [72.0, 585.4791666, 187.0281796, 590.101813], "content": "5.2 Quantitative Results", "level": 1}, {"type": "text", "bbox": [71.701, 598.652075, 539.9997875679996, 623.3293882], "content": "To enable valid cross-model comparisons with SmolDocling, we addressed multiple dataset and harmonization challenges:", "level": 1}, {"type": "list", "bbox": [99.387, 629.154075, 540.004223546, 664.7403882], "content": "Public evaluation datasets lack comprehensive multi-task annotations with diverse layouts. We augmented DocLayNet [70] with text content and reading-order to enable evaluation of both layout analysis and text recognition. Table structure, code listing, formula and chart recognition are evaluated on specific datasets.", "level": 2}, {"type": "list", "bbox": [99.387, 665.156075, 541.0947618512, 722.5603882], "content": "Different models each follow their own conventions for output markup (e.g., Markdown, HTML, DocTags) and produce annotations which do not fully align in semantics. For example, SmolDocling is trained to produce 14 distinct layout-class tags via DocTags , while Qwen2.5-VL combines standard HTML tags and a few explicit class attributes with only partially compatible definitions. GOT and Nougat output Markdown, which can express paragraphs, lists and tables without location annotation.", "level": 2}]}, {"page_number": 8, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 32.2990749999999, 500.8301544, 46.06738819999998], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion", "level": 1}, {"type": "list", "bbox": [99.387, 70.75507499999992, 540.0018646312003, 106.34138819999998], "content": "Input resolution of page images can affect model performance, but is rarely specified in literature. We standardized at 144 DPI (dots per inch), which reproduces enough detail to be perfectly legible by humans while keeping resource demand for inference reasonable.", "level": 2}, {"type": "text", "bbox": [71.671, 114.70507499999997, 541.097339771, 183.0183882], "content": "Text Recognition (OCR). We first evaluate text recognition accuracy of SmolDocling, compared to Nougat, GOT and Qwen2.5-VL (Table 2), to verify accurate character transcription and reading order. The text-similarity metrics, adopted from [12] require plain formatting without non-textual elements. For different tasks, we conduct evaluations on diverse content types: full-page documents (from DocLayNet test set) in Markdown format, code snippets (from our SynthCodeNet dataset) in plaintext, and equations (from Im2Latex-230k [21]) in LaTeX format. For Qwen2.5-VL and SmolDocling, we convert HTML or DocTags output to Markdown while preserving formatting where possible.", "level": 1}, {"type": "text", "bbox": [71.701, 185.63907499999993, 541.533651838, 243.04338819999998], "content": "Tables are omitted from the full-page output since they do not compare well through Markdown, due to loss of row and column spans. Code snippets must be reproduced with intact line breaks and indentation for accurate scoring. Formulas are evaluated separately in LaTeX format, which is natively produced by all the models tested in this work. The LaTeX outputs from all models, including the ground truth, undergo the same normalization procedure used for generating the SynthFormulaNet dataset. This ensures consistency and fairness in model comparisons.", "level": 1}, {"type": "text", "bbox": [71.631, 245.6640749999999, 540.2554303328002, 303.06838819999996], "content": "SmolDocling significantly outperforms GOT, Nougat and also Qwen2.5-VL, which is 27 times larger, on every metric in full-page transcription (see Table 2. For the code parsing task, we report results exclusively for SmolDocling, as we introduce this task for the first time, and the other models were not explicitly trained for it. SmolDocling also demonstrates superior performance compared to Nougat and Qwen2.5-VL in the formula recognition task, closely matching GOT's results.", "level": 1}, {"type": "text", "bbox": [71.631, 305.689075, 541.0942760316, 406.72938819999996], "content": "Layout. We evaluate element localization accuracy using the DocLayNet test set (Table 3), measuring the ability to reconstruct document representations that support content grounding and visual feature extraction. SmolDocling is compared against Qwen2.5-VL-7b [9], currently the only other VLM addressing this task. For a fair comparison, we map each model's label outputs and the ground-truth to six compatible classes: Text, Section Heading, List Item, Table, Picture, and Formula. Overall, SmolDocling outperforms Qwen2.5-VL-7b by a significant margin, however both models score far below a human baseline. This can be partially explained through inherent difficulty of the dataset, as indicated by relatively low human agreement scores. The scores are primarily impacted by moderate recall of element bounding boxes and different degrees of label confusion in both SmolDocling and Qwen2.5-VL. A majority of samples show accurate element bounding boxes as seen in the Appendix.", "level": 1}, {"type": "table", "bbox": [189.18585205078125, 418.41717529296875, 420.38201904296875, 509.1875915527344], "content": "|                |   SmolDocling |   Qwen2.5-VL |   Human |\n|----------------|---------------|--------------|---------|\n| Formula        |         0.267 |        0.059 |    0.84 |\n| Table          |         0.266 |        0.262 |    0.79 |\n| List Item      |         0.296 |        0.09  |    0.87 |\n| Section Header |         0.289 |        0.129 |    0.83 |\n| Text           |         0.204 |        0.18  |    0.85 |\n| Picture        |         0.066 |        0.078 |    0.7  |\n| Overall        |         0.231 |        0.133 |    0.82 |", "level": 1}, {"type": "text", "bbox": [71.701, 514.252075, 541.3262883968001, 538.9293882], "content": "Table 3: Layout analysis comparison. Evaluation of layout analysis task on DocLayNet with a subset of 6 class labels in mAP[0.5:0.95] metric. Additional reference points are provided for an estimate of human performance [70].", "level": 1}, {"type": "text", "bbox": [71.671, 556.015075, 541.5293487814, 635.2373881999999], "content": "Table Structure Recognition. We evaluate the accuracy of recovering table structures, i.e. shaping columns and rows with the correct text content and spans, through the TEDS metric [104], once including table text content and once on structure only with ommitted text (see Table 4). For isolated assessment, we provide cropped table images from FinTabNet [103] and PubTables-1M [78] test sets, which have poor image quality (72 dpi, compression artifacts). SmolDocling performs competitively against significantly larger models, despite challenges with text transcription from the low-resolution image crops, a limitation likely attributable to insufficient training at such resolutions. Scores for structure only TEDS evaluation are therefore much stronger.", "level": 1}, {"type": "text", "bbox": [71.651, 637.858075, 541.5319459369999, 695.2623882], "content": "Chart Extraction. In this task, we evaluate the ability to convert cropped sections of charts into datapoint tables. All table ouptut is transformed into HTML and compared against the ground-truth HTML using TEDS score, as reported in Table 5. We attribute SmolDocling's performance on charts to the variability in ground-truth styles and the differing interpretations of representations across datasets, which we observe leads to inconsistencies in the model's predictions. Despite its significantly smaller size, SmolDocling achieves a competitive TEDs score.", "level": 1}, {"type": "text", "bbox": [72.0, 697.883075, 539.9963230319994, 722.5603882], "content": "Overall, SmolDocling achieves leading performance across a wide range of document conversion tasks and outperforms significantly larger models in text recognition, layout analysis and structure extraction. Furthermore, the model's", "level": 1}]}, {"page_number": 9, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 32.2990749999999, 500.8301544, 46.06738819999998], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion", "level": 1}, {"type": "table", "bbox": [165.4056396484375, 69.8314208984375, 446.4163818359375, 130.56399536132812], "content": "| Method              | Model Size   | FinTabNet   | PubTables-1M   |\n|---------------------|--------------|-------------|----------------|\n| SmolVLM [56]        | 2.2B         | 0.18        | -              |\n| Granite Vision [81] | 2.98B        | 0.54        | -              |\n| TableFormer\u2020 [55]   | 52M          | 0.89        | 0.84           |\n| SmolDocling (Ours)  | 256M         | 0.52 (0.81) | 0.65 (0.88)    |", "level": 1}, {"type": "text", "bbox": [71.701, 135.439075, 541.0995813559996, 171.02538820000007], "content": "Table 4: Table structure and cell content reconstruction. Numbers are given in TEDS metric including text content, with structure-only score in brackets where available. We produced results for SmolDocling and TableFormer, other results are reflected here as published. \u2020TableFormer extracts the content through external OCR.", "level": 1}, {"type": "table", "bbox": [218.1203155517578, 187.74884033203125, 393.8341064453125, 257.6752624511719], "content": "| Model               | Size   | TEDs \u2191   |\n|---------------------|--------|----------|\n| Phi-3.5-vision      | 4B     | 0.40     |\n| Granite Vision [81] | 3B     | 0 . 95   |\n| SmolVLM [56]        | 2.2B   | 0.02     |\n| Molmo-E             | 1B     | 0.54     |\n| SmolDocling (Ours)  | 256M   | 0.75     |", "level": 1}, {"type": "text", "bbox": [71.701, 262.261075, 539.9999612536001, 286.93838819999996], "content": "Table 5: Chart analysis. We compare SmolDocling to other small-sized VLMs on the chart-to-table task using the TEDS metric. The results in this table are based on the work reported in [25].", "level": 1}, {"type": "text", "bbox": [72.0, 312.383075, 540.001702836, 347.96938819999997], "content": "high performance in downstream tasks, such as code listing and formula recognition, demonstrates its versatility and real-world applicability. Notably, we establish benchmark results for the code listing recognition, setting a standard for future evaluations.", "level": 1}, {"type": "section_header", "bbox": [72.0, 364.9181666, 180.56245219999997, 369.54081299999996], "content": "5.3 Qualitative Results", "level": 1}, {"type": "text", "bbox": [72.0, 378.091075, 541.5291686679997, 402.7683882], "content": "Several characteristics of SmolDocling output are not evident from task-based metrics alone. In this section, we highlight different output characteristics observed in practice. More illustrated samples can be found in the Appendix.", "level": 1}, {"type": "text", "bbox": [71.651, 405.388075, 540.3676190120005, 462.7933882], "content": "As presented in Figure 3, SmolDocling outputs a sequence of DocTags to encode content, structure and layout. A few notable traits beyond straight content reproduction include: 1) Labeling of page headers and footers, which allows ignoring these repetitive elements in downstream applications. 2) Linking of information, such as captions to tables or pictures, or list elements to parent list through nesting of tags. 3) Preserved line breaks and indentation for code snippets, while line wraps in paragraphs are correctly removed.", "level": 1}, {"type": "text", "bbox": [71.651, 465.413075, 540.2599334279995, 555.5453882], "content": "Among typical defects found in the output of SmolDocling are missing tags (e.g. absent location tags), malformed structure (such as missing closing tags), and endless repetition of tokens from some point in a page. Both of these phenomena are commonly seen in auto-regressive models and may prevent parsing or rendering the output correctly (see Appendix). Despite these limitations, SmolDocling demonstrates unique robustness advantages compared to ensemble systems like Docling, since conversion output is inferred in a single pass. In the latter, errors may accumulate throughout the ensemble model pipeline. For example, a mislocated table detection will lead to missing or distorted output of table structure and content, since this depends on correct table localization. In contrast, SmolDocling can reproduce the tables correctly, even with incorrectly determined location.", "level": 1}, {"type": "section_header", "bbox": [72.0, 575.4725632, 151.88464640000004, 581.019776], "content": "6 Conclusion", "level": 1}, {"type": "text", "bbox": [72.0, 592.551075, 540.0040938599999, 671.7743882], "content": "In this work, we introduce SmolDocling, an efficient and compact VLM optimized for document conversion while providing rich output representation. We also present a suite of new datasets with a unified format for document conversion, including the novel task of code listing transcription. We identify page element localization as a critical area requiring further refinement, where targeted techniques will significantly enhance performance in future iterations. Our results conclusively demonstrate that smaller models with unified, optimized output formats such as DocTags can effectively compete with significantly larger models, establishing a clear pathway for resource-efficient multi-task document understanding models.", "level": 1}]}, {"page_number": 10, "width": 612.0, "height": 792.0, "elements": [{"type": "section_header", "bbox": [72.0, 76.47556320000001, 131.5608064, 82.02277600000002], "content": "References", "level": 1}, {"type": "list", "bbox": [80.339, 93.3957999999999, 365.9494112, 105.78736479999998], "content": "Grobid. https://github.com/kermitt2/grobid , 2008-2025.", "level": 2}, {"type": "list", "bbox": [80.33900000000003, 107.87579999999991, 540.9909192319999, 150.15536480000003], "content": "Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Mart\u00edn Bl\u00e1zquez, Guilherme Penedo, Lewis Tunstall, Andr\u00e9s Marafioti, Hynek Kydl\u00ed\u010dek, Agust\u00edn Piqueres Lajar\u00edn, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Cl\u00e9mentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf. SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model, 2025.", "level": 2}, {"type": "list", "bbox": [80.339, 152.24379999999996, 539.7981920000002, 164.63536480000005], "content": "Amazon Web Services. Amazon Textract. https://aws.amazon.com/textract/ , 2024. Accessed: 2024-11-11.", "level": 2}, {"type": "list", "bbox": [80.339, 166.72479999999996, 540.0004906879999, 189.07836480000003], "content": "Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403 , 2023.", "level": 2}, {"type": "list", "bbox": [80.339, 191.16779999999994, 541.5122199999998, 213.52136480000001], "content": "Anthropic. Claude-3.5. https://www.anthropic.com/news/claude-3-5-sonnet , 2024. Accessed: 202402-11.", "level": 2}, {"type": "list", "bbox": [80.339, 215.60979999999995, 539.9981538559999, 247.92736480000008], "content": "Christoph Auer, Michele Dolfi, Andr\u00e9 Carvalho, Cesar Berrospi Ramis, and Peter WJ Staar. Delivering Document Conversion as a Cloud Service with High Throughput and Responsiveness. In 2022 IEEE 15th International Conference on Cloud Computing (CLOUD) , pages 363-373. IEEE, 2022.", "level": 2}, {"type": "list", "bbox": [80.339, 250.0157999999999, 541.3768627199998, 272.37036480000006], "content": "Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Nikolaos Livathinos, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Fabian Lindlbauer, Kasper Dinkla, et al. Docling Technical Report. arXiv preprint arXiv:2408.09869 , 2024.", "level": 2}, {"type": "list", "bbox": [80.339, 274.4588, 541.3797023359997, 296.81236479999995], "content": "Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 , 2023.", "level": 2}, {"type": "list", "bbox": [80.339, 298.9018, 540.9864360319997, 341.1813648], "content": "Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025.", "level": 2}, {"type": "list", "bbox": [76.169, 343.26980000000003, 540.2352310399996, 365.6233648], "content": "Ali Furkan Biten, Rub\u00e8n Tito, Lluis Gomez, Ernest Valveny, and Dimosthenis Karatzas. Ocr-idl: Ocr annotations for industry document library dataset. In European Conference on Computer Vision , pages 241-252. Springer, 2022.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 367.7128, 301.5125648, 380.1043648], "content": "Lukas Blecher. LaTeX-OCR, 2024. Accessed: 2024-11-09.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 382.1928, 540.002732288, 404.54736479999997], "content": "Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint arXiv:2308.13418 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 406.6358, 540.0005803519997, 428.9893648], "content": "Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. OneChart: Purify the Chart Structural Extraction via One Auxiliary Token. arXiv preprint arXiv:2404.09987 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 431.0788, 540.985909312, 463.3953648], "content": "Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research , 25(70):1-53, 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 465.48380000000003, 428.3512592, 477.8753648], "content": "Pyecharts Contributors. pyecharts: Python library for Echarts, 2024. Accessed: 2024-11-06.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 479.96380000000005, 428.8713104000001, 492.3553648], "content": "Pygments Contributors. Pygments: Python syntax highlighter, 2024. Accessed: 2024-11-06.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 494.4448, 539.9986974079999, 516.7983648], "content": "Brian Davis, Bryan Morse, Brian Price, Chris Tensmeyer, Curtis Wigington, and Vlad Morariu. End-to-end document recognition and understanding with dessurt. In European Conference on Computer Vision , pages 280-296. Springer, 2022.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 518.8878, 539.998697408, 541.2413648], "content": "Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M Rush. Image-to-markup generation with coarse-to-fine attention. In International Conference on Machine Learning , pages 980-989. PMLR, 2017.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 543.3298, 539.9986974079998, 585.6093648], "content": "Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model. arXiv preprint arXiv:2401.16420 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 587.6988, 540.990919232, 629.9783648], "content": "Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Songyang Zhang, Haodong Duan, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Zhe Chen, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Kai Chen, Conghui He, Xingcheng Zhang, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD. arXiv preprint arXiv:2404.06512 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 632.0668000000001, 541.1488080000003, 654.4203648], "content": "Gregory Eritsyan. Im2LaTeX 230K. https://www.kaggle.com/datasets/gregoryeritsyan/ im2latex-230k , 2023. Accessed: 2025-03-06.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 656.5098, 340.1846480000002, 668.9013648], "content": "Hugging Face. latex-formulas Dataset, 2024. Accessed: 2024-11-09.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 670.9898, 539.9986974079993, 693.3443648], "content": "Philippe Gervais, Asya Fadeeva, and Andrii Maksai. Mathwriting: A dataset for handwritten mathematical expression recognition. URL https://arxiv. org/abs/2404.10690 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 695.4328, 518.6881920000001, 707.8243648], "content": "Google Cloud. Document AI. https://cloud.google.com/document-ai , 2024. Accessed: 2024-11-11.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 709.9128000000001, 298.3474256000002, 722.3043648], "content": "IBM Granite Team. Granite 3.0 Language Models, 2024.", "level": 2}]}, {"page_number": 11, "width": 612.0, "height": 792.0, "elements": [{"type": "list", "bbox": [76.169, 71.87579999999991, 540.3096521599998, 94.22936479999998], "content": "Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 95.7177999999999, 541.3797023359997, 118.07236480000006], "content": "Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403.12895 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 119.5598, 541.0578713407995, 141.91436480000004], "content": "Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding. arXiv preprint arXiv:2409.03420 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 143.40279999999996, 539.9975048768002, 165.75636480000003], "content": "Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia , pages 4083-4091, 2022.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 167.24479999999994, 488.55241599999994, 179.63636480000002], "content": "J. D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science & Engineering , 9(3):90-95, 2007.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 181.12479999999994, 539.9982270592, 213.44136479999997], "content": "Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) , volume 2, pages 1-6. IEEE, 2019.", "level": 2}, {"type": "list", "bbox": [76.169, 214.9297999999999, 540.986799744, 247.24636480000004], "content": "Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 248.73479999999995, 539.9979800959998, 271.0883648], "content": "Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, \u00c1kos K\u00e1d\u00e1r, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300 , 2017.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 272.57679999999993, 541.3797023359998, 294.9313648], "content": "Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. Chart-to-text: A large-scale benchmark for chart summarization. arXiv preprint arXiv:2203.06486 , 2022.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 296.4188, 540.9860601600001, 328.7363648], "content": "Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Donut: Document understanding transformer without ocr. arXiv preprint arXiv:2111.15664 , 7(15):2, 2021.", "level": 2}, {"type": "list", "bbox": [76.169, 330.2238, 540.3174529279999, 362.54036479999996], "content": "Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton. PubChem 2019 update: improved access to chemical data. Nucleic Acids Research , 47(D1):D1102-D1109, 10 2018.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 364.0288, 540.9886238336001, 396.34536479999997], "content": "Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015-4026, 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 397.8338, 540.0005086208004, 430.1503648], "content": "Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Mu\u00f1oz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. The Stack: 3 TB of permissively licensed source code. Preprint , 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 431.6388, 540.9841227008004, 463.9553648], "content": "Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 465.4438, 514.3211024000002, 477.8353648], "content": "Greg Landrum. RDKit: Open-Source Cheminformatics Software. http://www.rdkit.org/. Accessed: 1 January 2023.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 479.3238, 540.0026515903999, 501.67736479999996], "content": "Hugo Lauren\u00e7on, Andr\u00e9s Marafioti, Victor Sanh, and L\u00e9o Tronchon. Building and better understanding vision-language models: insights and future directions., 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 503.1658, 540.9828046399998, 525.5203647999999], "content": "Hugo Lauren\u00e7on, L\u00e9o Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models?, 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 527.0078, 540.9864360319997, 559.3253648], "content": "Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In International Conference on Machine Learning , pages 18893-18912. PMLR, 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 560.8127999999999, 540.002037312, 593.1303648], "content": "David Lewis, Gady Agam, Shlomo Argamon, Ophir Frieder, David Grossman, and Jefferson Heard. Building a test collection for complex document information processing. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval , pages 665-666, 2006.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 594.6178, 539.9979800960001, 616.9723648], "content": "Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 618.4608000000001, 541.0596018559999, 640.8143648], "content": "Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 642.3027999999999, 541.3817571200002, 664.6573648], "content": "Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.", "level": 2}, {"type": "list", "bbox": [76.169, 666.1448, 540.0028559360002, 688.4993648], "content": "Han Lin, Peng Yang, and Fanlong Zhang. Review of scene text detection and recognition. Archives of computational methods in engineering , 27(2):433-454, April 2020.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 689.9878, 540.000370432, 722.3043648], "content": "Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. arXiv preprint arXiv:2212.10505 , 2022.", "level": 2}]}, {"page_number": 12, "width": 612.0, "height": 792.0, "elements": [{"type": "list", "bbox": [76.169, 71.87579999999991, 541.3807334720001, 104.19236480000006], "content": "Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. arXiv preprint arXiv:2212.09662 , 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 106.11379999999997, 514.5452624000002, 118.50536480000005], "content": "Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved Baselines with Visual Instruction Tuning, 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 120.42679999999996, 479.63431999999995, 132.81836480000004], "content": "Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. In NeurIPS , 2023.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 134.73879999999997, 540.9861491072, 177.01836480000009], "content": "Nikolaos Livathinos, Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Kasper Dinkla, Yusik Kim, Shubham Gupta, Rafael Teixeira de Lima, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, and Peter W. J. Staar. Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion, 2025.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 178.9398, 540.2295373759998, 201.29436480000004], "content": "Nam Ly., Atsuhiro Takasu., Phuc Nguyen., and Hideaki Takeda. Rethinking Image-Based Table Recognition Using Weakly Supervised Methods. pages 872-880, 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 203.21579999999994, 540.1696418240002, 235.53236479999998], "content": "Maksym Lysak, Ahmed Nassar, Nikolaos Livathinos, Christoph Auer, and Peter Staar. Optimized Table Tokenization for Table Structure Recognition. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 37-50, Cham, 2023. Springer Nature Switzerland.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 237.4537999999999, 540.9864360319999, 269.77036480000004], "content": "Andr\u00e9s Marafioti, Orr Zohar, Miquel Farr\u00e9, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf. SmolVLM: Redefining small and efficient multimodal models. 2025.", "level": 2}, {"type": "list", "bbox": [76.169, 271.69179999999994, 539.9980876927997, 294.0463648], "content": "Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. arXiv preprint arXiv:2305.14761 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 295.9668, 539.9979800959997, 318.32136479999997], "content": "Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244 , 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 320.2428, 539.9974638336, 342.5973648], "content": "Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 2200-2209, 2021.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 344.5178, 541.5168882559998, 376.8353648], "content": "Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. Chartassisstant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. arXiv preprint arXiv:2401.02384 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 378.7568, 540.0001900287999, 401.11036479999996], "content": "Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. PlotQA: Reasoning over Scientific Plots. In The IEEE Winter Conference on Applications of Computer Vision (WACV) , March 2020.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 403.0318, 541.1434800000002, 425.3863648], "content": "Microsoft Azure. AI Document Intelligence. https://azure.microsoft.com/en-us/products/ ai-services/ai-document-intelligence , 2024. Accessed: 2024-11-11.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 427.3078, 540.1724662399996, 459.62436479999997], "content": "Lucas Morin, Martin Danelljan, Maria Isabel Agea, Ahmed Nassar, Valery Weber, Ingmar Meijer, Peter Staar, and Fisher Yu. MolGrapher: Graph-based Visual Recognition of Chemical Structures. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 19552-19561, October 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 461.5458, 539.9986974079997, 483.8993648], "content": "Lucas Morin, Val\u00e9ry Weber, Gerhard Ingmar Meijer, Fisher Yu, and Peter W. J. Staar. PatCID: an open-access dataset of chemical structures in patent documents. Nature Communications , 15(1):6532, Aug 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 485.8208, 541.5111042560003, 508.17536479999995], "content": "OpenAI. Hello GPT-4O. https://openai.com/index/hello-gpt-4o/ , 2024. Accessed: 2024-02-09, 2024-0211, 2024-02-12.", "level": 2}, {"type": "list", "bbox": [76.169, 510.0968, 381.7752543999999, 522.4883648], "content": "R OpenAI. Gpt-4 technical report. arxiv 2303.08774. View in Article , 2(5), 2023.", "level": 2}, {"type": "list", "bbox": [76.16899999999993, 524.4097999999999, 540.9843916927996, 546.7633648], "content": "Vik Paruchuri. Marker: Convert PDF to Markdown Quickly with High Accuracy. https://github.com/VikParuchuri/marker, 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 548.6848, 503.72719199999995, 561.0763648], "content": "Vik Paruchuri. Texify. https://github.com/VikParuchuri/texify , 2024. Accessed: 2024-11-11.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 562.9978, 539.9986974079997, 585.3513648], "content": "Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 587.2728, 540.0046710144004, 619.5903648], "content": "Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S. Nassar, and Peter Staar. DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 3743-3751, New York, NY, USA, 2022. Association for Computing Machinery.", "level": 2}, {"type": "list", "bbox": [76.169, 621.5108, 540.1751023616002, 653.8283648], "content": "Ruchir Puri, David Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, Shyam Ramji, Ulrich Finkler, Susan Malaika, and Frederick Reiss. CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks, 2021.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 655.7498, 540.0010901248, 688.0663648], "content": "Edward O. Pyzer-Knapp, Matteo Manica, Peter Staar, Lucas Morin, Patrick Ruch, Teodoro Laino, John R. Smith, and Alessandro Curioni. Foundation models for materials discovery - current state and future directions. npj Computational Materials , 11(1):61, Mar 2025.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 689.9878, 540.9879833024002, 722.3043648], "content": "Yujie Qian, Jiang Guo, Zhengkai Tu, Zhening Li, Connor W. Coley, and Regina Barzilay. MolScribe: Robust Molecular Structure Recognition with Image-to-Graph Generation. Journal of Chemical Information and Modeling , 63(7):1925-1934, Apr 2023.", "level": 2}]}, {"page_number": 13, "width": 612.0, "height": 792.0, "elements": [{"type": "list", "bbox": [76.169, 71.87579999999991, 540.986553216, 104.19236480000006], "content": "Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research , 21(140):1-67, 2020.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 105.98079999999993, 541.3829840384004, 138.29736480000008], "content": "Kohulan Rajan, Henning Otto Brinkhaus, M. Isabel Agea, Achim Zielesny, and Christoph Steinbeck. DECIMER.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications. Nature Communications , 14(1):5045, Aug 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 140.08579999999995, 539.99857792, 172.4023648], "content": "Johannes Rausch, Octavio Martinez, Fabian Bissig, Ce Zhang, and Stefan Feuerriegel. Docparser: Hierarchical document structure parsing from renderings. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 4328-4338, 2021.", "level": 2}, {"type": "list", "bbox": [76.169, 174.19079999999985, 540.3096521599997, 196.54536480000002], "content": "Felix M Schmitt-Koopmann, Elaine M Huang, Hans-Peter Hutter, Thilo Stadelmann, and Alireza Darvishy. MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition. IEEE Access , 2024.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 198.33379999999988, 540.9838650879999, 230.65036480000003], "content": "Brandon Smock, Rohith Pesala, and Robin Abraham. PubTables-1M: Towards comprehensive table extraction from unstructured documents. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4624-4632, 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 232.4387999999999, 541.3775503999998, 264.75636480000003], "content": "Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mohit Bansal. Unifying vision, text, and layout for universal document processing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19254-19264, 2023.", "level": 2}, {"type": "list", "bbox": [76.169, 266.5447999999999, 540.98615584, 298.8613648], "content": "Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 300.6498, 540.9903902143997, 392.7423648], "content": "Granite Vision Team, Leonid Karlinsky, Assaf Arbelle, Abraham Daniels, Ahmed Nassar, Amit Alfassi, Bo Wu, Eli Schwartz, Dhiraj Joshi, Jovana Kondic, Nimrod Shabtay, Pengyuan Li, Roei Herzig, Shafiq Abedin, Shaked Perek, Sivan Harary, Udi Barzelay, Adi Raz Goldfarb, Aude Oliva, Ben Wieles, Bishwaranjan Bhattacharjee, Brandon Huang, Christoph Auer, Dan Gutfreund, David Beymer, David Wood, Hilde Kuehne, Jacob Hansen, Joseph Shtok, Ken Wong, Luis Angel Bathen, Mayank Mishra, Maksym Lysak, Michele Dolfi, Mikhail Yurochkin, Nikolaos Livathinos, Nimrod Harel, Ophir Azulai, Oshri Naparstek, Rafael Teixeira de Lima, Rameswar Panda, Sivan Doveh, Shubham Gupta, Subhro Das, Syed Zawad, Yusik Kim, Zexue He, Alexander Brooks, Gabe Goodhart, Anita Govindjee, Derek Leist, Ibrahim Ibrahim, Aya Soffer, David Cox, Kate Soule, Luis Lastras, Nirmit Desai, Shila Ofek-koifman, Sriram Raghavan, Tanveer Syeda-Mahmood, Peter Staar, Tal Drory, and Rogerio Feris. Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence, 2025.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 394.5308, 540.9889107584005, 426.8473648], "content": "Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 428.6358, 540.9902019199999, 460.9523648], "content": "Micha\u0142 Turski, Tomasz Stanis\u0142awek, Karol Kaczmarek, Pawe\u0142 Dyda, and Filip Grali\u0144ski. CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data. In Gernot A. Fink, Rajiv Jain, Koichi Kise, and Richard Zanibbi, editors, Document Analysis and Recognition - ICDAR 2023 , pages 348-365, Cham, 2023. Springer Nature Switzerland.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 462.7408, 540.9886238336003, 485.09536479999997], "content": "Unstructured.io Team. Unstructured.io: Open-Source Pre-Processing Tools for Unstructured Data. https://unstructured.io, 2024. Accessed: 2024-11-19.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 486.8838, 544.0174160000003, 509.2383648], "content": "Vicuna. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. https://vicuna.lmsys. org/ , 2023.", "level": 2}, {"type": "list", "bbox": [76.169, 511.0268, 540.9900225920003, 543.3433648], "content": "Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. MinerU: An Open-Source Solution for Precise Document Content Extraction, 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 545.1318, 491.18020479999996, 557.5233648], "content": "Michael L Waskom. Seaborn: statistical data visualization. Journal of Open Source Software , 6(60):3021, 2021.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 559.3118, 540.9899867263999, 591.6283648], "content": "Maurice Weber, Carlo Siebenschuh, Rory Butler, Anton Alexandrov, Valdemar Thanner, Georgios Tsolakis, Haris Jabbar, Ian Foster, Bo Li, Rick Stevens, et al. WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data. Advances in Neural Information Processing Systems , 36, 2024.", "level": 2}, {"type": "list", "bbox": [76.16899999999998, 593.4168, 541.3849199680001, 615.7713648], "content": "Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model. arXiv preprint arXiv:2409.01704 , 2024.", "level": 2}, {"type": "list", "bbox": [76.169, 617.5598, 540.9827459583997, 649.8763648], "content": "Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824-24837, 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 651.6648, 539.9986974079999, 674.0193648], "content": "David Weininger. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences , 28(1):31-36, Feb 1988.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 675.8078, 542.8689760000003, 698.1613648], "content": "Wes McKinney and the Pandas Development Team. Pandas: powerful Python data analysis toolkit, Release 1.4.4. https: //pandas.pydata.org/pandas-docs/version/1.4/pandas.pdf , 2022.", "level": 2}, {"type": "list", "bbox": [76.16900000000004, 699.9498, 541.3797023359994, 722.3043648], "content": "Renqiu Xia, Bo Zhang, Haoyang Peng, Hancheng Ye, Xiangchao Yan, Peng Ye, Botian Shi, Junchi Yan, and Yu Qiao. StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding. arXiv preprint arXiv:2309.11268 , 2023.", "level": 2}]}, {"page_number": 14, "width": 612.0, "height": 792.0, "elements": [{"type": "list", "bbox": [76.169, 71.87579999999991, 540.9861491072, 104.19236480000006], "content": "Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, et al. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185 , 2024.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 105.74879999999996, 540.9909192320001, 138.0653648], "content": "Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, et al. Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 , 2020.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 139.6217999999999, 540.3183327104, 171.93836480000004], "content": "Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining , pages 1192-1200, 2020.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 173.49479999999994, 541.3833827200001, 205.81136479999998], "content": "Zuoyu Yan, Xiaode Zhang, Liangcai Gao, Ke Yuan, and Zhi Tang. ConvMath: a convolutional sequence network for mathematical expression recognition. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 4566-4572. IEEE, 2021.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 207.36779999999987, 541.3791105536002, 239.68436480000003], "content": "Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model. arXiv preprint arXiv:2310.05126 , 2023.", "level": 2}, {"type": "list", "bbox": [76.16900000000001, 241.24079999999992, 540.9909192319999, 283.51936479999995], "content": "Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition. arXiv preprint arXiv:2309.15112 , 2023.", "level": 2}, {"type": "list", "bbox": [72.0, 285.07579999999996, 541.5167374719999, 337.3183648], "content": "Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. InternLMXComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output. arXiv preprint arXiv:2407.03320 , 2024.", "level": 2}, {"type": "list", "bbox": [72.00000000000001, 338.8738, 539.9980428799997, 361.22836479999995], "content": "Qiming Zhang, Jing Zhang, Yufei Xu, and Dacheng Tao. Vision Transformer with Quadrangle Attention. arXiv preprint arXiv:2303.15105 , 2023.", "level": 2}, {"type": "list", "bbox": [72.00000000000001, 362.78479999999996, 540.9888390272004, 395.1013648], "content": "Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. URL https://arxiv. org/abs/2205.01068 , 3:19-0, 2023.", "level": 2}, {"type": "list", "bbox": [72.0, 396.6578, 540.2352310399997, 428.9743648], "content": "Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 697-706, 2021.", "level": 2}, {"type": "list", "bbox": [72.0, 430.5308, 541.3810203968, 452.88436479999996], "content": "Xu Zhong, Elaheh ShafieiBavani, and Antonio Jimeno Yepes. Image-based table recognition: data, model, and evaluation. arXiv preprint arXiv:1911.10683 , 2019.", "level": 2}, {"type": "list", "bbox": [72.0, 454.44079999999997, 541.5165744320003, 476.79536479999996], "content": "Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 , 2023.", "level": 2}]}, {"page_number": 15, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [58.00420054962188, 249.85692688935512, 87.10481144181726, 256.6014063866289], "content": "Multi Task", "level": 1}, {"type": "text", "bbox": [258.3648255468455, 26.208333328273852, 288.8023255468455, 30.458333328273852], "content": "PubTablelM", "level": 1}, {"type": "text", "bbox": [251.2863364542636, 65.87500019436243, 296.58866203565896, 70.83333320980432], "content": "SynthFormulaNet", "level": 1}, {"type": "picture", "bbox": [75.61947631835938, 73.98613739013672, 538.412353515625, 321.9443054199219], "content": "", "level": 1}, {"type": "text", "bbox": [72.0, 331.49807500000003, 541.0974002959998, 367.0843882], "content": "Figure 3: Training datasets. A treemap visualization of dataset sizes contributing to the training of SmolDocling, categorized by type. Each rectangle represents a dataset, with its area proportional to the total dataset size. Each color indicates a dataset type. 'Multi-Task\" refers to datasets that include multiple different tasks.", "level": 2}, {"type": "section_header", "bbox": [72.00000000000003, 395.32356319999997, 240.0422912000001, 400.870776], "content": "A Additional Datasets Details", "level": 1}, {"type": "section_header", "bbox": [72.00000000000003, 418.8831666, 165.22004820000006, 423.50581299999993], "content": "A.1 SynthChartNet", "level": 1}, {"type": "text", "bbox": [72.0, 432.056075, 541.0956473522003, 489.46038819999995], "content": "SynthChartNet includes four distinct types of charts: line, pie, bar, and stacked bar, generated using three visualization libraries: Matplotlib [30], Seaborn [87], and Pyecharts [15]. Each data set was rendered through all three libraries, producing a final dataset of 2.5 million visually diverse charts. The rendering process was designed to leverage the full range of features available in these libraries, ensuring visual variety while maintaining the plausibility of the data presented. Figure 4 depicts 10 samples from SynthChartNet.", "level": 1}, {"type": "section_header", "bbox": [72.0, 506.2541666, 162.20138040000003, 510.87681299999997], "content": "A.2 SynthCodeNet", "level": 1}, {"type": "text", "bbox": [71.631, 519.4270750000001, 541.5354849564001, 609.5583882], "content": "To render the code snippets of SynthCodeNet, we employed two rendering libraries: LaTeX and Pygments [16]. LaTex, in conjunction with its listings package, enabled the generation of code renderings similar to those commonly seen in academic and technical publications, while Pygments facilitated renderings with aesthetics resembling those of IDEs. The rendering process utilized the complete set of features provided by these libraries to enhance visual diversity while preserving the plausibility of the presented data. This included randomizing various stylistic parameters such as font, font size, style, weight, line spacing, tab size, background color, line numbering (including randomizing the starting line number), and programming language-specific syntax highlighting. Figure 5 shows 4 samples from SynthCodeNet.", "level": 1}, {"type": "section_header", "bbox": [72.0, 626.3531666, 177.58363480000003, 630.975813], "content": "A.3 SynthFormulaNet", "level": 1}, {"type": "text", "bbox": [71.701, 639.525075, 539.9997875679999, 664.2023882], "content": "The extracted formulas of SynthFormulaNet underwent a rigorous normalization procedure to ensure the model was trained only on correct and standardized code. The normalization process consisted of the following major steps:", "level": 1}, {"type": "list", "bbox": [96.06, 670.448075, 539.9949081919997, 695.1253882], "content": "Token Filtering and Removal : Unnecessary or redundant LaTeX tokens were removed or replaced based on predefined policies. This included:", "level": 2}, {"type": "list", "bbox": [119.313, 696.071075, 250.4007181999999, 709.8393882], "content": "Eliminating unwanted tokens.", "level": 2}, {"type": "list", "bbox": [119.313, 708.792075, 350.7838758000001, 722.5603882], "content": "Replacing tokens with equivalent ones for consistency.", "level": 2}, {"type": "text", "bbox": [453.02325558744184, 14.166666695000004, 475.67441837813953, 19.833333361666632], "content": "ChartQA", "level": 1}, {"type": "text", "bbox": [322.07122104525797, 16.291666524999982, 475.67441871967657, 27.666666524999982], "content": "MolGrapher Synthetic 300k WordScape-PocLayNe[nichart", "level": 1}, {"type": "text", "bbox": [406.3052327421802, 47.458333305, 424.7093025096221, 53.124999971666625], "content": "PlotQA", "level": 1}, {"type": "text", "bbox": [356.7141590301681, 72.69643520177078, 390.7742125881003, 78.88689794998038], "content": "SynthDocNet", "level": 1}, {"type": "text", "bbox": [443.1061719932873, 47.42391144883425, 470.01882733443955, 53.15942195705372], "content": "FinTabNet", "level": 1}]}, {"page_number": 16, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [13.02816935271395, 239.89534886028537, 19.107981084688234, 244.24127895977267], "content": "600", "level": 1}, {"type": "text", "bbox": [13.028169404052155, 250.32558144739778, 20.84507029313093, 254.67151154688509], "content": "500 -", "level": 1}, {"type": "text", "bbox": [13.028169404052155, 261.62500002307615, 20.84507029313093, 265.97093012256346], "content": "400-", "level": 1}, {"type": "text", "bbox": [13.028169404052155, 272.00000005634377, 20.84507029313093, 276.401162680703], "content": "300 -", "level": 1}, {"type": "text", "bbox": [13.028168890065082, 283.35465122297427, 19.107981257822242, 286.8313952389858], "content": "200", "level": 1}, {"type": "text", "bbox": [13.028169404052155, 293.7848837440063, 20.84507029313093, 298.1308138434936], "content": "100-", "level": 1}, {"type": "text", "bbox": [20.84507055193949, 308.56104662162267, 28.661971441018267, 312.9069767211099], "content": "1700", "level": 1}, {"type": "text", "bbox": [33.00469495349818, 138.20058151733042, 51.24412951363815, 139.99999996673228], "content": "borond suel", "level": 1}, {"type": "text", "bbox": [37.34741781411653, 308.56104662162267, 44.2957744425345, 312.9069767211099], "content": "1800", "level": 1}, {"type": "text", "bbox": [53.84976513419654, 308.561046547121, 59.9295775019537, 312.90697664660826], "content": "1900", "level": 1}, {"type": "text", "bbox": [28.661972857750055, 354.62790699582473, 63.403755999839895, 360.0000000375626], "content": "Property plant and equipment -", "level": 1}, {"type": "text", "bbox": [15.63380455492809, 378.9651164260286, 38.215962961503614, 381.6666665918408], "content": "Iccumulated depre..", "level": 1}, {"type": "text", "bbox": [18.2394363852778, 411.9941861275335, 38.2159620098709, 416.3401162270208], "content": "Cross asset volur", "level": 1}, {"type": "text", "bbox": [34.74178531013805, 434.5930233038673, 61.66666629158336, 439.8081394868304], "content": "Construction in progress in millions except per share", "level": 1}, {"type": "text", "bbox": [17.370892648125775, 468.333333275986, 66.8779320361466, 471.99999994265266], "content": "Construction in progress in millions except per share", "level": 1}, {"type": "text", "bbox": [18.239437892297673, 478.0000000378618, 45.164318873742985, 482.3982557039588], "content": "Total revenues", "level": 1}, {"type": "text", "bbox": [17.370894112770014, 488.3333334044628, 72.0892016078872, 492.8284882620983], "content": "Income (loss) from continuing", "level": 1}, {"type": "text", "bbox": [17.370892505614297, 499.66666661211815, 59.92957272208565, 502.3895347367621], "content": "Income from continuing", "level": 1}, {"type": "text", "bbox": [17.370892676313833, 508.3333333490166, 50.37558729708206, 512.8197673318658], "content": "Net income (loss)", "level": 1}, {"type": "text", "bbox": [18.23943684493173, 518.9040698452161, 39.08450673018563, 522.3808138612277], "content": "Total assets", "level": 1}, {"type": "text", "bbox": [17.37089222506978, 528.3333333804891, 46.90140726006327, 532.8110464885518], "content": "Short-term debt", "level": 1}, {"type": "text", "bbox": [18.239437433630446, 538.8953489363392, 67.74647682165129, 543.333333402892], "content": "Total debt (including capital", "level": 1}, {"type": "text", "bbox": [17.370892003758797, 549.3255814763706, 63.4037543491364, 553.671511575858], "content": "Total stockholders' equity", "level": 1}, {"type": "text", "bbox": [69.4835682804702, 308.561046534704, 76.43192490888815, 312.90697663419127], "content": "2000", "level": 1}, {"type": "text", "bbox": [87.72300642794632, 259.8866280289583, 107.69953078097366, 261.66666670363077], "content": "Enteray Texal", "level": 1}, {"type": "text", "bbox": [79.03755917450457, 266.8401164010513, 117.25351681610613, 268.66666664633635], "content": "-** nteray New or eani", "level": 1}, {"type": "text", "bbox": [85.98591496832148, 273.66666666851484, 117.25351852463659, 275.66666666851484], "content": "Enteray Hississippl", "level": 1}, {"type": "text", "bbox": [86.85446182700736, 279.8779070987257, 106.8309861800347, 281.66666664818456], "content": "Interay Art.", "level": 1}, {"type": "text", "bbox": [84.2488264204841, 308.56104662162267, 92.06572730956287, 312.9069767211099], "content": "2100", "level": 1}, {"type": "text", "bbox": [100.7511740169494, 308.561046534704, 111.17370895957636, 312.90697663419127], "content": "2200", "level": 1}, {"type": "text", "bbox": [99.01408463644653, 354.6279070827433, 106.8309855255253, 359.84302326570634], "content": "-Ind", "level": 1}, {"type": "text", "bbox": [114.64788754444534, 308.56104654556884, 123.3333333299678, 312.9069766450562], "content": "2300", "level": 1}, {"type": "text", "bbox": [88.59154956993636, 535.4186046452661, 96.40845045901513, 539.7645347447534], "content": "12.31", "level": 1}, {"type": "text", "bbox": [123.33333431862148, 377.99999996423276, 145.91549272519697, 383.33333329756607], "content": "band improvements", "level": 1}, {"type": "text", "bbox": [123.33333339708996, 411.9941861565064, 148.52112585721363, 416.3401162559937], "content": "Duldings and building", "level": 1}, {"type": "text", "bbox": [97.27699582278147, 435.33333348798425, 128.54459937909658, 440.67732556231493], "content": "- machinery and equipmen", "level": 1}, {"type": "text", "bbox": [97.27699556185534, 519.7732558370295, 105.9624413473778, 524.1191859365168], "content": "13.85", "level": 1}, {"type": "text", "bbox": [100.7511739830819, 558.0174418545683, 108.56807487216065, 562.3633719540557], "content": "16.92", "level": 1}, {"type": "text", "bbox": [244.92957795607091, 240.66666664263994, 278.8028130228027, 242.66666664263994], "content": "Total Cash Dividends Paid", "level": 1}, {"type": "text", "bbox": [245.7981272601436, 246.84883733128402, 303.1220662656774, 248.66666670178256], "content": "Toral Cash Widends Paid der Common Share", "level": 1}, {"type": "text", "bbox": [310.93896710989117, 61.71220941232025, 317.88732373830914, 66.05813951180755], "content": "2011", "level": 1}, {"type": "text", "bbox": [310.9389675261179, 66.92732560447143, 318.7558684151967, 71.27325570395874], "content": "2012", "level": 1}, {"type": "text", "bbox": [311.80751191349054, 72.14244189440603, 318.7558685419085, 76.48837199389334], "content": "2013", "level": 1}, {"type": "text", "bbox": [310.9389671961708, 79.09593035453975, 317.88732318880585, 80.8343022035998], "content": "2014", "level": 1}, {"type": "text", "bbox": [310.9389680394999, 83.33333330465962, 336.12676049962363, 87.78779058767975], "content": "Total unrecognized", "level": 1}, {"type": "text", "bbox": [239.7183127509903, 91.26453501469723, 336.12676001661526, 97.34883728113607], "content": "2500 5000 7500 10000 12500 15000 17500 20000", "level": 1}, {"type": "picture", "bbox": [129.58473205566406, 107.56428527832031, 283.44012451171875, 188.4427490234375], "content": "", "level": 1}, {"type": "text", "bbox": [296.1737090919946, 192.95930235952233, 309.20187808816974, 197.30523245900963], "content": "1Year", "level": 1}, {"type": "text", "bbox": [232.33247440982026, 175.57558091405565, 252.74968467084727, 196.00145371679002], "content": "United States loYear", "level": 1}, {"type": "text", "bbox": [297.91079828311393, 198.17441856578603, 312.6760558006107, 203.38953474874904], "content": "United States loYear", "level": 1}, {"type": "text", "bbox": [295.30516442239406, 205.99709313325047, 310.07042193989076, 210.34302323273778], "content": "\u00b7 5Year|", "level": 1}, {"type": "picture", "bbox": [129.50840759277344, 212.6746826171875, 263.2164306640625, 297.2315368652344], "content": "", "level": 1}, {"type": "text", "bbox": [227.55868335635873, 299.8691861934705, 335.25820728213995, 302.4767441260063], "content": "O..................... . ... . . ... ..... ........ .... \u00b7", "level": 1}, {"type": "text", "bbox": [237.98122091396803, 305.0843023486575, 246.6666666994905, 309.43023244814475], "content": "2075", "level": 1}, {"type": "text", "bbox": [251.00939011266735, 305.0843023486575, 260.56338015885063, 309.43023244814475], "content": "2100", "level": 1}, {"type": "text", "bbox": [264.906103543076, 305.0843023486575, 273.59154932859843, 309.43023244814475], "content": "2125", "level": 1}, {"type": "text", "bbox": [277.06572800755976, 305.0843023486575, 287.4882629501867, 309.43023244814475], "content": "2150", "level": 1}, {"type": "text", "bbox": [290.96244175585986, 305.0843023486575, 300.5164318020431, 309.43023244814475], "content": "2175", "level": 1}, {"type": "text", "bbox": [304.85915523760667, 305.0843023486575, 315.2816901802336, 309.43023244814475], "content": "2200", "level": 1}, {"type": "text", "bbox": [316.1502358243637, 305.95348843213327, 328.3098592883122, 308.56104636466904], "content": "2225", "level": 1}, {"type": "picture", "bbox": [129.56288146972656, 317.75054931640625, 252.27435302734375, 399.7124328613281], "content": "", "level": 1}, {"type": "text", "bbox": [237.1126759214618, 401.5639535005834, 243.192488289219, 405.9098836000707], "content": "First econd", "level": 1}, {"type": "text", "bbox": [237.1126758517264, 410.2558139897885, 249.27230058724072, 411.9941859977944], "content": "First econd", "level": 1}, {"type": "text", "bbox": [237.1126761972231, 415.4709302266613, 245.79812198274556, 419.8168603261486], "content": "Third", "level": 1}, {"type": "picture", "bbox": [129.63485717773438, 419.8803405761719, 273.7579345703125, 527.4857788085938], "content": "", "level": 1}, {"type": "text", "bbox": [117.25352151672821, 559.7558140095329, 125.07042240580698, 563.2325580255446], "content": "7.69", "level": 1}, {"type": "picture", "bbox": [130.27662658691406, 550.5965576171875, 281.7286071777344, 675.2907104492188], "content": "", "level": 1}, {"type": "text", "bbox": [219.088, 682.047075, 392.9120981999998, 695.8153882], "content": "Figure 4: Samples from SynthChartNet.", "level": 2}, {"type": "picture", "bbox": [343.7693786621094, 95.26988220214844, 466.8607177734375, 185.6379852294922], "content": "", "level": 1}, {"type": "picture", "bbox": [343.62298583984375, 198.2246551513672, 477.6539001464844, 303.7454833984375], "content": "", "level": 1}, {"type": "picture", "bbox": [343.8607482910156, 317.83782958984375, 464.3912048339844, 399.9420166015625], "content": "", "level": 1}, {"type": "picture", "bbox": [344.0517272949219, 416.71923828125, 476.05810546875, 525.5697021484375], "content": "", "level": 1}, {"type": "picture", "bbox": [344.1797180175781, 567.9732055664062, 481.4819641113281, 668.5801391601562], "content": "", "level": 1}, {"type": "text", "bbox": [307.46478903819997, 354.62790703049745, 322.23004655569673, 358.97383712998476], "content": "24.7%", "level": 1}, {"type": "text", "bbox": [330.04694879811336, 305.0843023486575, 339.60093884429665, 309.43023244814475], "content": "2250", "level": 1}, {"type": "text", "bbox": [327.4413149067933, 392.00290698133836, 341.3380281636293, 397.2180231643013], "content": "25.3%", "level": 1}, {"type": "text", "bbox": [296.1737093308934, 212.0000000307681, 310.07042258772935, 217.33333336410146], "content": "\u00b7 3Year", "level": 1}, {"type": "text", "bbox": [270.11737102411547, 372.0116278681116, 284.8826285416122, 377.22674405107455], "content": "24.4%", "level": 1}, {"type": "text", "bbox": [288.3568079224428, 411.1249999901076, 302.2535211792787, 416.3401161730706], "content": "25.7%", "level": 1}]}, {"page_number": 17, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [11.325581386500724, 137.19180085869732, 17.69622092138444, 140.72767196652228], "content": "16..", "level": 1}, {"type": "text", "bbox": [11.325581315715844, 164.06442172004438, 17.696220850599563, 167.60029282786923], "content": "20.I", "level": 1}, {"type": "text", "bbox": [23.359012733438266, 23.317241517320895, 64.41424275018578, 28.994143539847073], "content": "#pragma once", "level": 1}, {"type": "text", "bbox": [23.359013845833243, 49.50219619495692, 156.43459524118208, 56.627586256960285], "content": "#include <sdm/world/solvable_by_dp. hpp>", "level": 1}, {"type": "text", "bbox": [23.359013362136597, 55.866764281662654, 196.78197847841565, 62.95655186682245], "content": "#include <sdm/utils/value_function/initializer.hpp>", "level": 1}, {"type": "text", "bbox": [24.066860429723835, 69.9517242616713, 67.95348833670057, 75.6676427842101], "content": "namespace sdm namespace", "level": 1}, {"type": "text", "bbox": [38.223836912005815, 84.15373358613965, 67.95348807479651, 88.39677891552947], "content": "namespace sdm namespace", "level": 1}, {"type": "text", "bbox": [37.51598833670058, 89.10395309265664, 41.0552325227471, 94.7613468651765], "content": "{", "level": 1}, {"type": "text", "bbox": [51.67296607015904, 96.88286969826265, 70.78488340411418, 101.12591502765247], "content": "class", "level": 1}, {"type": "text", "bbox": [50.96511624367732, 102.5402634879714, 54.504360429723825, 108.19765726049127], "content": "{", "level": 1}, {"type": "text", "bbox": [71.47373394574048, 96.79374300943755, 97.70213727461125, 102.62939027937716], "content": "registry protected:", "level": 1}, {"type": "text", "bbox": [51.67296535517804, 109.61200582486754, 84.94186070401524, 115.26939959738729], "content": "registry protected:", "level": 1}, {"type": "text", "bbox": [65.12209531196697, 115.9200001573463, 202.44476973057158, 122.34114175877733], "content": "typedef std:: map<std:: string, std:: shared", "level": 1}, {"type": "text", "bbox": [202.4447672413033, 116.68374847870439, 220.14098817153584, 123.04831615511694], "content": "_ptr<", "level": 1}, {"type": "text", "bbox": [80.69476637300873, 129.41288441094525, 187.57994079161338, 136.57241375162346], "content": "world, Config config) > map_type;", "level": 1}, {"type": "text", "bbox": [77.15552446800508, 122.24896541628402, 232.1744198168423, 130.24344817490464], "content": "Initializer > (*) (std:: shared_ptr<SolvableByDP>", "level": 1}, {"type": "text", "bbox": [65.12209389390988, 136.48462665355135, 152.18750087065408, 143.5675861923728], "content": "static map_type container;", "level": 1}, {"type": "text", "bbox": [71.49273301824127, 82.73938507355251, 108.30087255312499, 87.68960462450741], "content": "initializer sane: Stage the app", "level": 1}, {"type": "text", "bbox": [254.82558079367735, 19.653103463867296, 290.92587149135176, 23.336749644908082], "content": "initializer sane: Stage the app", "level": 1}, {"type": "text", "bbox": [259.78052411408066, 70.61793095216035, 328.44186132338297, 77.0819910212773], "content": "A20RE, NEBAT NAVE: Y0218431-T5E-APP...", "level": 1}, {"type": "text", "bbox": [259.78052597759995, 74.96046857994031, 288.8023257617893, 77.08199108579913], "content": "................", "level": 1}, {"type": "text", "bbox": [262.61191671705427, 88.9386207025301, 375.86773067054264, 93.34699853014149], "content": "1t: contains (gathub. event.pull_request labela.*.name, 'stage*)", "level": 1}, {"type": "text", "bbox": [262.61191840409396, 103.95461206563368, 274.64534863665205, 107.49048317345853], "content": "steps:", "level": 1}, {"type": "text", "bbox": [266.8590124537306, 107.49048315157802, 316.4084310583817, 111.02635425940298], "content": "- uses: actions/checkout (vl", "level": 1}, {"type": "text", "bbox": [266.15116393505326, 110.2572413134867, 334.81250114435556, 115.2693996376297], "content": "- name: pm install and build webpack run:", "level": 1}, {"type": "text", "bbox": [270.3982557821003, 115.26939971256547, 278.1845929914026, 118.09809659882535], "content": "- name: pm install and build webpack run:", "level": 1}, {"type": "text", "bbox": [273.93749974340477, 118.80527091477347, 294.4651160224746, 122.34114202259843], "content": "npm instal.", "level": 1}, {"type": "text", "bbox": [266.8590127250726, 125.16983895155693, 332.6889545855377, 129.57724136666798], "content": "- uses: actions/upload-artifact\u00eanain apm run build", "level": 1}, {"type": "text", "bbox": [273.93749969031614, 122.24896551073743, 298.7122089926417, 125.91310344177191], "content": "- uses: actions/upload-artifact\u00eanain apm run build", "level": 1}, {"type": "text", "bbox": [270.3982562967922, 129.41288432288115, 280.30813938237446, 132.948755430706], "content": "with:", "level": 1}, {"type": "text", "bbox": [273.2296515757025, 136.48462662930217, 298.0043608780281, 140.72767195869199], "content": "path: public/", "level": 1}, {"type": "text", "bbox": [273.9375007196463, 132.90827594019788, 317.1162797894138, 136.57241387123236], "content": "name: webpack artifacts", "level": 1}, {"type": "text", "bbox": [259.78052383389047, 144.26354318086055, 294.46511685714626, 147.7994142886855], "content": "Busld-Docker-Image:", "level": 1}, {"type": "text", "bbox": [262.6119205713985, 147.79941433752242, 304.37499942535527, 151.33528544534738], "content": "runs-on: ubuntu-latest", "level": 1}, {"type": "code", "bbox": [22.65116186164609, 149.72993673669635, 459.39389659642075, 466.7349925872483], "content": "public: static std:: vector<std:: string > available (): static std:: shared_ptr<Initializer > make(std :: string name, std:: shared_ptr<SolvableByDP> world, Config config = {}); } :1 template <class TInitializer > std:: shared_ptr<Initializer > createlnstance (std:: shared_ptr<SolvableByDP> world, Config config) return std:: make_shared<TInitializer >(world, config): package br.com.biot. integra caopagarmeapi.modulos.jwt.utids import br.com.biot. integracaopagarmeapi.config.exception.ValidacaoExceptipn import io.jsonwebtoken.Claims import lombok.extern.slf4j.Slf4j public class JwtCampoUtil{ public static String getCampold(Claims claims) {| tryi return (String) claims.get(\"id\"); loge to tro do tentar recuperar I do JWT: 0'; claims toString): throw new ValidacaoException(\"Erro ao tentar recuperar o campo ID do Token.\"): public static String getCampo(String campo, Claims claims) { return (String) claims.get(campo); sagetro euro de entar recuperar o campo f] do IT: 0', campo, claims toString): throw new ValidacaoException( String.format(\"Erro ao tentar recuperar o campo %s do Token.\", campo) 29| needs: build name: Build image and store in GitHub Packages steps: - name: Checkout uses: actions/checkout iva - name: Download built artifact uses: actions/download-arts factinain path: public - name: create inage and store in Packages uses: nattdavsa0351/actions/docker-gpr01.3.0 with: repo-token: 3{ (secrets.GITHUB,TOKEN) ) runneone wountuel tent needs: Build-Docker-Image name: Deploy app container to Azure steps: - name: \"Login via Azure CLI\" MIRA creda: 91{ SeCretA.AZURE_CREDENTIALS 1) - uses: azure/docker-logan@va with: logan-server: #{(env. IMAGE_REGISTRY_URL)) def main (filename, days) : \u00b7 array I of buckets 0-8, each representing with that counter value fish_by_counter = [O for i in range (9)] for counter in read_counters (filename): fish_by_counter [counter) += 1 the number of fish for i in range (days): fish_by_counter \u00b7 spawn (fish_by_counter) print (f\"After {days) days: {sum(fish_by_counter)} fish\") def spawn (fish_by_counter): # rotate left res = fish_by_counter (1:) + fish_by_counter [:1] # fish that were in bucket O are now new fish in bucket 8. restore the original fish into bucket 6. res [6] += res [8] return res def read_counters (filename) : with open (filename) as \u00a3 : return [int (counter) split (',')] for counter in f. readline ().strip(). -_name.. == '__main_.': main('input.txt', 80) main ('input.txt', 256)", "level": 1}, {"type": "text", "bbox": [220.597, 624.777075, 391.4024303999998, 638.5453882], "content": "Figure 5: Samples from SynthCodeNet.", "level": 2}]}, {"page_number": 18, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [9.217484046511995, 24.32122105004828, 31.00426471745388, 31.03052308008739], "content": "kejnKm kejnKm", "level": 1}, {"type": "text", "bbox": [99.71641773335632, 23.482558071443805, 121.5031984042982, 31.030523172945777], "content": "kejnKm kejnKm", "level": 1}, {"type": "text", "bbox": [106.42004271821776, 65.41569780541647, 128.20682338915967, 72.12499983545558], "content": "kejnKm", "level": 1}, {"type": "text", "bbox": [101.35118329488006, 73.58318610317087, 145.00702653177095, 82.40809295782242], "content": "=I+II + II.", "level": 1}, {"type": "formula", "bbox": [7.189023801093627, 105.78333390119383, 239.63414228171135, 129.92494172975262], "content": "", "level": 1}, {"type": "text", "bbox": [15.083155932210529, 145.91570454474459, 31.842217302672207, 153.57794472950206], "content": "d \u2264 1", "level": 1}, {"type": "text", "bbox": [243.00639668464342, 51.99709312070718, 249.71002135987692, 58.70639515074629], "content": "Y", "level": 1}, {"type": "text", "bbox": [368.6993602653209, 114.89680252367225, 382.1066096157879, 121.60610455371148], "content": "4.30", "level": 1}, {"type": "text", "bbox": [368.6993602653209, 134.18604652400677, 382.1066096157879, 140.056686117999], "content": "4.31", "level": 1}, {"type": "text", "bbox": [368.6993605851883, 145.92732569561997, 382.9445629406539, 152.63662772565908], "content": "4.32", "level": 1}, {"type": "formula", "bbox": [47.76332576177219, 200.44040741765934, 381.26865631721375, 228.95494182172956], "content": "", "level": 1}, {"type": "formula", "bbox": [119.61103057861328, 276.1187744140625, 490.6849365234375, 330.5036315917969], "content": "", "level": 1}, {"type": "formula", "bbox": [119.65660858154297, 371.4732971191406, 237.5425567626953, 390.8725891113281], "content": "", "level": 1}, {"type": "text", "bbox": [368.6993602149464, 394.171511854554, 380.4307035554161, 402.558139392103], "content": "(x)", "level": 1}, {"type": "formula", "bbox": [353.61620457413386, 538.4215117141714, 381.2686562800665, 547.6468023231831], "content": "", "level": 1}, {"type": "formula", "bbox": [118.35175323486328, 615.8834838867188, 492.0532531738281, 664.5174560546875], "content": "", "level": 1}, {"type": "text", "bbox": [212.906, 671.676075, 399.0936847999997, 685.4443882], "content": "Figure 6: Samples from SynthFormulaNet.", "level": 1}]}, {"page_number": 19, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 32.2990749999999, 500.8301544, 46.06738819999998], "content": "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion", "level": 1}, {"type": "list", "bbox": [96.06, 70.75507499999992, 407.29196640000004, 84.5233882], "content": "Structural Normalization : The equation structure was standardized by:", "level": 2}, {"type": "list", "bbox": [119.31299999999999, 84.94907499999988, 280.517658, 98.71738819999996], "content": "Normalizing spacing between tokens.", "level": 2}, {"type": "list", "bbox": [119.31299999999999, 109.3520749999999, 392.1984040000001, 123.12038819999998], "content": "Ensuring consistent use of tokens that require one or two braces.", "level": 2}, {"type": "list", "bbox": [119.31299999999999, 97.1500749999999, 284.9410523999999, 110.91838819999998], "content": "Standardizing left and right delimiters.", "level": 2}, {"type": "list", "bbox": [96.05999999999997, 123.54607499999986, 445.2093002000001, 137.31438819999994], "content": "Token Simplification : Redundant tokens and patterns were simplified, including:", "level": 2}, {"type": "list", "bbox": [119.31299999999997, 137.74007499999982, 498.58900940000007, 151.5083881999999], "content": "Collapsing consecutive redundant tokens (e.g., repeated primes, dots, or excessive spacing).", "level": 2}, {"type": "list", "bbox": [119.31299999999997, 149.94207499999982, 486.2257620000001, 163.7103881999999], "content": "Converting special constructs such as \\Big into \\left and \\right for consistency.", "level": 2}, {"type": "text", "bbox": [71.631, 168.915075, 541.0995813559998, 269.9563882], "content": "The extracted equations were subsequently rendered using LaTex at 120 dpi. Several aspects of the rendering procedure have been randomized to ensure the visual diversity of the dataset. Each formula was rendered using a randomly selected font from a set of 85 possible options, with variations in font size, style, weight, and line spacing. Additionally, with a certain probability, each formula was enclosed within an \"align\" environment, allowing for the inclusion of a randomly generated equation number on either the left or right side of the equation. The equation number itself was chosen at random from a diverse set of possibilities, including small and large numbers, integers, floating-point values, or letters. Furthermore, it could be enclosed in single or double parentheses, brackets, or curly braces and was sometimes preceded by text labels such as 'Eqn.\", 'Eqn\", 'Eq.\", or 'Equation\". Figure 6 depicts 6 samples from SynthFormulaNet.", "level": 1}, {"type": "section_header", "bbox": [72.0, 286.3041666, 187.22743159999996, 290.9268129999999], "content": "A.4 DocTags vocabulary", "level": 1}, {"type": "text", "bbox": [72.0, 299.47707499999996, 486.26076880000005, 313.2453881999999], "content": "Here we provide a reference to a complete vocabulary of DocTags for SmolDocling , and their meaning.", "level": 1}, {"type": "text", "bbox": [71.522, 315.866075, 540.0015441792, 351.4523882], "content": "We represent tags in an XML-like style of notation. Some tags have an opening and closing part, and wrap around textual content (or other tags) for example: <text>hello world</text> . Other tags are standalone and do not have a closing tag; they mark a special instruction, for example: <page_break>", "level": 1}, {"type": "text", "bbox": [71.522, 354.072075, 541.534087192, 378.7493882], "content": "The complete DocTags snippet can represent a page or multiple pages, wrapped into <doctag>...</doctag> . When content represents multiple pages, we use <page_break> tag as a separator.", "level": 1}, {"type": "text", "bbox": [70.725, 381.370075, 541.6834670079998, 427.8653882], "content": "Within the scope of <doctag> we place high-level, tags that wrap around the textual content of certain document blocks and identify the type of the block. These tags are: <text> , <caption> , <footnote> , <formula> , <title> , <page_footer> , <page_header> , <picture> , <section_header> , <document_index> , <code> , <otsl> , <list_item> , <ordered_list> , <unordered_list>", "level": 1}, {"type": "text", "bbox": [72.0, 430.486075, 541.0945402803999, 476.98138819999997], "content": "Each element can nest additional standalone location tags that encode its position on the page as a bounding box, represented in DocTags as <loc_ x1 ><loc_ y1 ><loc_ x2 ><loc_ y2 > . Every x,y pair of coordinates belongs to a fixed grid. In our case we used integer values in the range from 0 to 500 which proportionally maps to width and height of the page.", "level": 1}, {"type": "text", "bbox": [70.725, 479.602075, 541.4596795330002, 547.9153882], "content": "<otsl> element contains table representation with following OTSL tags, that are standalone and used to describe structure of a table: <fcel> , <ecel> , <lcel> , <ucel> , <xcel> , <nl> . These tags follow rules as described in [55], we additionally differentiate OTSL notion of a cell into <fcel> - full cell, or cell which contain content, and <ecel> - empty cell. We also augment tabular structure information with extra tags <ched> , <rhed> , <srow> that replace <fcel> to describe cells which belong to column and row headers of the table, as well as table section accordingly. We mark such headers when ground truth about table headers from the underlying dataset is available.", "level": 1}, {"type": "text", "bbox": [70.725, 550.535075, 540.1865449603999, 575.2133882], "content": "<list_item> elements are placed within <ordered_list> or <unordered_list> and define weather it's enumerated (ordered) list or not.", "level": 1}, {"type": "text", "bbox": [70.725, 577.833075, 540.1890668959998, 613.4193882], "content": "<picture> and <otsl> elements by themselves can encapsulate <caption> tag if appropriate picture or table has it's own caption. This way we can connect extra descriptive information to illustrations and tables in the document.", "level": 1}, {"type": "text", "bbox": [70.725, 616.040075, 541.6869581199998, 695.2623882], "content": "<code> elements contain pieces of code as a content, and as such, respect tabulation and line breaks. Code elements also contains special standalone classification tag -<_ programming-language _> where programminglanguage is an appropriate programming language name. Supported values (57 in total) for programming language names are: Ada , Awk , Bash , bc , C , C# , C++ , CMake , COBOL , CSS , Ceylon , Clojure , Crystal , Cuda , Cython , D , Dart , dc , Dockerfile , Elixir , Erlang , FORTRAN , Forth , Go , HTML , Haskell , Haxe , Java , JavaScript , Julia , Kotlin , Lisp , Lua , Matlab , MoonScript , Nim , OCaml , ObjectiveC , Octave , PHP , Pascal , Perl , Prolog , Python , Racket , Ruby , Rust , SML , SQL , Scala , Scheme , Swift , TypeScript , unknown , VisualBasic , XML , YAML", "level": 1}, {"type": "text", "bbox": [71.522, 697.883075, 541.53460744, 722.5603882], "content": "Within <picture> elements we include picture classification with extra standalone tags < image_class> . We classify images from our datasets into following categories: natural_image , pie_chart , bar_chart , line_chart ,", "level": 1}]}, {"page_number": 20, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [72.0, 70.75507499999992, 541.09589542, 95.43238819999999], "content": "flow_chart , scatter_chart , heatmap , remote_sensing , chemistry_molecular_structure , chemistry_markush_structure , icon , logo , signature , stamp , qr_code , bar_code , screenshot , map , stratigraphic_chart , cad_drawing , electrical_diagram", "level": 1}, {"type": "section_header", "bbox": [72.0, 115.35956320000003, 317.2370176, 120.90677600000004], "content": "B Additional Details about the SmolDocling", "level": 1}, {"type": "table", "bbox": [71.04452514648438, 140.9374237060547, 540.4243774414062, 366.4342041015625], "content": "| Instruction                 | Description                                                               |\n|-----------------------------|---------------------------------------------------------------------------|\n| Full conversion             | Convert this page to docling.                                             |\n| Chart                       | Convert chart to table (e.g., <chart> ).                                  |\n| Formula                     | Convert formula to LaTeX (e.g., <formula> ).                              |\n| Code                        | Convert code to text (e.g., <code> ).                                     |\n| Table                       | Convert table to OTSL (e.g., <otsl> ).                                    |\n| No-Code Ac- tions/Pipelines | OCR the text in a specific location: <loc_155><loc_233><loc_206><loc_237> |\n| No-Code Ac- tions/Pipelines | Identify element at: <loc_247><loc_482><loc_252><loc_486>                 |\n| No-Code Ac- tions/Pipelines | Find all 'text' elements on the page, retrieve all section headers.       |\n| No-Code Ac- tions/Pipelines | Detect footer elements on the page.                                       |", "level": 1}, {"type": "text", "bbox": [208.308, 371.043075, 403.6916159999997, 384.81138819999995], "content": "Table 6: Supported Conversion Instructions.", "level": 2}]}, {"page_number": 21, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [25.953342156928077, 25.124710864372787, 69.97212313179601, 33.40160495465102], "content": "Input Image", "level": 1}, {"type": "text", "bbox": [19.98447212740683, 41.89952149784688, 29.310559083928567, 47.22009566052634], "content": "OH", "level": 1}, {"type": "text", "bbox": [19.98447210964286, 90.44976073894736, 25.97981372454969, 95.77033490162682], "content": "Cl", "level": 1}, {"type": "text", "bbox": [2.6729231175909687e-06, 108.40669852718304, 427.66770453627714, 118.38277508220699], "content": "\u00b7 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR", "level": 1}, {"type": "text", "bbox": [2.9393826912347486e-06, 119.71291867691389, 235.15062405739513, 129.02392346160286], "content": "2.5-VL-72B and SmolDocling on a simple molecule image.", "level": 1}, {"type": "text", "bbox": [130.48750151180852, 23.16666681337938, 180.53750015652474, 32.0769227520052], "content": "GOT-OCR 2.0", "level": 1}, {"type": "text", "bbox": [180.52639763210402, 63.18181827326555, 191.1847827252717, 70.49760774694971], "content": "NH,", "level": 1}, {"type": "picture", "bbox": [91.59849548339844, 71.18405151367188, 520.1006469726562, 146.24417114257812], "content": "", "level": 1}, {"type": "text", "bbox": [72.0, 155.3040749999999, 541.0942576439995, 179.98138820000008], "content": "Figure 7: Qualitative comparison for molecule image recognition. Predictions are shown for GOT-OCR 2.0, Qwen2.5-VL-72B and SmolDocling on a simple molecule image.", "level": 2}, {"type": "section_header", "bbox": [72.0, 195.6685632000001, 263.42666240000005, 201.21577600000012], "content": "C Additional Qualitative Analysis", "level": 1}, {"type": "section_header", "bbox": [72.0, 219.38316659999998, 281.52344059999996, 224.005813], "content": "C.1 Molecule Image Recognition Experiment", "level": 1}, {"type": "text", "bbox": [72.0, 232.555075, 541.6807994399998, 289.96038819999995], "content": "Extracting molecule images from documents has potential to accelerate research in chemistry and materials discovery [72]. Some recent document understanding models, such as GOT-OCR 2.0 [89] and Qwen2.5-VL [9], claim to perform molecule image recognition in documents. Following this direction, we experimented with training SmolDocling for molecule image recognition on full document pages and cropped images. In order to predict molecule structures (graphs) with SmolDocling, we use the molecule string identifier named SMILES [91].", "level": 1}, {"type": "text", "bbox": [72.0, 292.580075, 540.0005216860001, 349.9853882], "content": "Datasets. For training on cropped images, we use the MolGrapher-Synthetic-300K dataset [63]. The dataset contains 300K synthetic samples. It is created using real chemical-structures retrieved from the database PubChem [36] which are then synthetically drawn using the library RDKit [40]. For training of full pages, we use Doc2Mol-80K, a subset of the PatCID dataset [64]. Our subset contains 80 000 real patent documents published in the United-States between 2022 and 2024.", "level": 1}, {"type": "text", "bbox": [72.0, 352.605075, 540.0019407599998, 420.91938819999996], "content": "Evaluation. Figure 7 shows an example of predicted molecules for a simple molecule image. While some models capture high-level molecular features, they consistently fail to reconstruct its detailed structure. Notably, this molecule is trivial for specialized models such as MolGrapher [63], MolScribe [73] or DECIMER [75], which accurately recognize its structure. Reducing the gap between general document understanding models and specialized models may require incorporating a specialized vocabulary for SMILES tokens, explicitly encoding atoms and bonds, and extending the SMILES sequence with localization tokens for each atom.", "level": 1}, {"type": "section_header", "bbox": [72.0, 437.8671666, 206.00693259999994, 442.48981299999997], "content": "C.2 Layout analysis samples", "level": 1}, {"type": "text", "bbox": [71.701, 451.040075, 541.6806475521998, 486.62638819999995], "content": "To illustrate and compare the visual grounding capabilities of SmolDocling, a set of sample predictions with SmolDocling and Qwen2.5-VL from DocLayNet [70] is shown in Table 7. Note that the element location results are independent from the correct reproduction of document content and structure.", "level": 1}]}, {"page_number": 22, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [50.82412872101403, 8.380720828427002e-08, 114.87645347498888, 7.664991462591388], "content": "Ground-Truth", "level": 1}, {"type": "text", "bbox": [17.40552351395739, 16.72361827195516, 30.633720686013547, 23.691792252667938], "content": "46\"", "level": 1}, {"type": "text", "bbox": [20.886628434093495, 73.00000000327748, 52.912790811080924, 77.34673357337635], "content": "peR, heath and safety", "level": 1}, {"type": "text", "bbox": [20.88662774459137, 160.96482420262748, 36.20348856431644, 165.14572859105522], "content": "Final gonn so yoR and convertible loan", "level": 1}, {"type": "text", "bbox": [20.88662840121628, 165.84254609343145, 55.69767479464125, 169.3333332816435], "content": "Final gonn so yoR and convertible loan", "level": 1}, {"type": "text", "bbox": [50.82412833291056, 238.31155783991153, 102.34447374223491, 240.4020100341254], "content": "tor save did exit) in the Opoon promp pur puse the Enter key:", "level": 1}, {"type": "text", "bbox": [50.82412854637895, 240.3333332730091, 76.58430125104113, 243.99999993967572], "content": "tor save did exit) in the Opoon promp pur puse the Enter key:", "level": 1}, {"type": "text", "bbox": [51.520350162505075, 246.66666665781634, 84.93895327702978, 249.4606364479572], "content": "trollowing display appears.", "level": 1}, {"type": "text", "bbox": [54.30523208130554, 255.00000001087926, 79.36918568788055, 258.6666666775459], "content": "apposoices, tress later, yange Sample form nurter 3 in the Tet \"description' prompt.", "level": 1}, {"type": "text", "bbox": [50.82412759723644, 314.2646567071668, 119.05377710518611, 317.74874369752325], "content": "apposoices, tress later, yange Sample form nurter 3 in the Tet \"description' prompt.", "level": 1}, {"type": "text", "bbox": [51.52035159625617, 324.6666666761549, 84.24273307133073, 326.8073700363945], "content": "Hollowing display appears", "level": 1}, {"type": "text", "bbox": [54.305235342714866, 346.31825805828595, 78.67296476847676, 347.71189269548285], "content": "Tyle doices, tress toler", "level": 1}, {"type": "text", "bbox": [55.697676265060466, 350.333333319755, 75.19186056807888, 351.9999999864217], "content": "Create earlay..", "level": 1}, {"type": "text", "bbox": [50.824128268238844, 385.99999992381845, 91.20494015317594, 389.66666659048514], "content": "Moneyle Actior: Press the Enter key.", "level": 1}, {"type": "text", "bbox": [65.44476794081878, 155.33333330095297, 95.38226794081878, 160.33333330095297], "content": "310.000,000 pal during the 2015", "level": 1}, {"type": "text", "bbox": [105.82558257432225, 114.97487440411112, 131.58575527898444, 119.15577879253874], "content": "Sign untain Change in matters suusequent to", "level": 1}, {"type": "text", "bbox": [105.82558199564608, 135.8793970287901, 132.97819797920852, 140.06030141721772], "content": "Sign untain Change in matters suusequent to", "level": 1}, {"type": "text", "bbox": [105.82558085817534, 139.9999999380641, 139.24418651406305, 143.66666660473072], "content": "the end of the financial year", "level": 1}, {"type": "text", "bbox": [174.0552327105734, 18.117253048652856, 181.71366248509517, 23.69179223322317], "content": "40", "level": 1}, {"type": "text", "bbox": [209.56250041773254, 2.2623123641096754e-07, 269.4375004177325, 9.755443799229283], "content": "SmolDocling", "level": 1}, {"type": "text", "bbox": [368.9618290540748, -0.19149229828849457, 424.7300322058149, 10.643753469940634], "content": "2wen2.5-VL", "level": 1}, {"type": "text", "bbox": [182.49, 68.42807499999992, 472.54291496399986, 256.74238820000005], "content": "Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. Win following, display appears we want yourse", "level": 1}, {"type": "text", "bbox": [421.90988355747317, 259.9128979355612, 437.9229647459669, 264.7906197220602], "content": "Mum: F4 To 11S", "level": 1}, {"type": "picture", "bbox": [70.63162994384766, 265.6536560058594, 584.84765625, 692.060791015625], "content": "", "level": 1}]}, {"page_number": 23, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [50.718022541995225, -7.913274657767033e-08, 115.33139534472961, 7.639399131479308], "content": "Ground-Truth", "level": 1}, {"type": "text", "bbox": [34.043610828626576, 28.333333303083236, 97.2674420053844, 29.99999996974998], "content": "301 Workers engarod in cential oe emeriency works pass ntial or emergency works\" includes-", "level": 1}, {"type": "text", "bbox": [36.1279081177575, 50.00000005921004, 86.8459307821455, 53.4757929440367], "content": "301 Workers engarod in cential oe emeriency works pass ntial or emergency works\" includes-", "level": 1}, {"type": "text", "bbox": [41.68604582632101, 54.17028378088651, 115.33139596466735, 56.99999998648343], "content": "gunmpletions, maintenance, repairs, and asset replacement activities.", "level": 1}, {"type": "text", "bbox": [124.3633724342539, 60.42070118408708, 134.09011613237405, 64.58764600576274], "content": "Quality", "level": 1}, {"type": "text", "bbox": [40.99127799126094, 58.333333382459955, 134.0901142542818, 64.66666671579333], "content": "*/ Supie (England) Regularions 20164), the Water Supply Nu Qual", "level": 1}, {"type": "text", "bbox": [36.12790762047312, 71.53255429608362, 133.39534333010855, 73.66666665417688], "content": "Low crane Ixcince tacans the looser of a seucrare Ixcice under sochon 178% of tho", "level": 1}, {"type": "text", "bbox": [40.29651199913738, 72.9215358669585, 70.86627772586264, 77.08848068863415], "content": "Water Industry Act 1991(e):", "level": 1}, {"type": "text", "bbox": [36.115944446583725, 77.61524438893412, 134.09590457028165, 80.6824142141702], "content": "we crape services\" has the meaning givee in sction 2191) of the Water Industry Actl", "level": 1}, {"type": "text", "bbox": [36.127905462325934, 84.66666655276663, 134.09011452808113, 90.33333321943337], "content": "48/ We industy ice 19 lar the mating given in seties 17ACT) and 2I9C) of the nak managemcat on bchall of", "level": 1}, {"type": "text", "bbox": [31.264535409722253, 95.83973302397044, 63.918600859754804, 97.99999997966404], "content": "48/ We industy ice 19 lar the mating given in seties 17ACT) and 2I9C) of the nak managemcat on bchall of", "level": 1}, {"type": "text", "bbox": [36.12790698153958, 99.31218704572404, 71.56104551099922, 102.78464111676897], "content": "pay one nvironment Agency, or)", "level": 1}, {"type": "text", "bbox": [36.127907657259584, 104.17362269363139, 81.98255751891323, 105.66666667264042], "content": "tal tood dinory to three.", "level": 1}, {"type": "text", "bbox": [34.04360797372343, 107.64607686103227, 79.20348956552031, 109.72954911292436], "content": "Cle the purposes of sub-paragraph l wyneed and \"coastal crosion\" have the meanings given in section I of the Plood and", "level": 1}, {"type": "text", "bbox": [36.12790464597427, 111.66666666916922, 134.0901137117295, 114.00000000250259], "content": "Cle the purposes of sub-paragraph l wyneed and \"coastal crosion\" have the meanings given in section I of the Plood and", "level": 1}, {"type": "text", "bbox": [40.296512721171325, 113.20200333687603, 75.72965125063095, 116.67445740792095], "content": "Water Management Act 20100h);", "level": 1}, {"type": "text", "bbox": [36.12791062776764, 121.53589313008683, 113.2470919428718, 125.00834720113176], "content": "pg/w9, management\" has the meaning gives in section 3 of that Acci.", "level": 1}, {"type": "text", "bbox": [34.043611207886364, 127.00000000269006, 96.5726741147874, 129.33333333602343], "content": "229,n29 1) Workers engaged in essential or emergency works- we we stricity interccenector,", "level": 1}, {"type": "text", "bbox": [40.99127898192719, 138.2036728322571, 76.42441751138682, 141.67612690330213], "content": "229,n29 1) Workers engaged in essential or emergency works- we we stricity interccenector,", "level": 1}, {"type": "text", "bbox": [40.29651418534994, 142.99999997075645, 133.3953504483708, 145.14858099874107], "content": "regree trict heat network as delined in regulation 2 of the Heat Network (Meicnng and", "level": 1}, {"type": "text", "bbox": [45.15988455530637, 144.99999992380503, 76.4244185518884, 148.66666659047178], "content": "Billing) Regulations 2014(j).", "level": 1}, {"type": "text", "bbox": [45.159884621265576, 151.99999995333462, 74.34011635140881, 155.66666662000125], "content": "Billing) Regulations 2014.", "level": 1}, {"type": "text", "bbox": [40.29651284785757, 159.66666672561132, 134.78488565059195, 163.33333339227806], "content": "put mao ommissicaling, maintenance and repair of industrial machinery for use on al", "level": 1}, {"type": "text", "bbox": [31.959303412935295, 190.98497493206708, 84.76162580063061, 195.15191975374262], "content": "\u0420\u0434 (1 4/3130. There as Do trialt areading envie:", "level": 1}, {"type": "text", "bbox": [30.569773998584065, 236.66666667011623, 93.09883690548509, 238.33333333678286], "content": "I canations of the saddle point are thie to lowing)", "level": 1}, {"type": "text", "bbox": [132.70058158016943, 243.07178626219593, 138.25872074541195, 246.54424033324096], "content": "(11))", "level": 1}, {"type": "text", "bbox": [27.095931503807815, 250.66666666264746, 138.2587224380526, 254.3333333293141], "content": "Mll9. C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on the support of p. We can prove that these equations lead to a singular support for p. We", "level": 1}, {"type": "text", "bbox": [26.401165054807166, 255.57262110146053, 138.95348235623945, 259.73956592313607], "content": "Mll9. C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on the support of p. We can prove that these equations lead to a singular support for p. We", "level": 1}, {"type": "text", "bbox": [26.40116375534269, 261.0000000446156, 57.66569775192472, 264.66666671128223], "content": "do this by contradiction.", "level": 1}, {"type": "text", "bbox": [27.095930939964486, 275.6666666961572, 50.02325587079131, 279.33333336282385], "content": "woRd then that", "level": 1}, {"type": "text", "bbox": [27.095932016668144, 290.99165272941906, 98.65697734544419, 294.464106800464], "content": "397199e first two terms obviously are cancelled. Notice that", "level": 1}, {"type": "text", "bbox": [26.401163564160942, 305.57595990702526, 75.03488141897866, 309.7429047287009], "content": "3'iMat away from 2' - O we have that", "level": 1}, {"type": "text", "bbox": [55.5813963897756, 313.2153589400804, 110.46802358704122, 317.38230376175596], "content": "91a + 03) - P\u00aa (log(=*) + log(=*)) - 0|", "level": 1}, {"type": "text", "bbox": [27.095928400513316, 320.8547579036864, 138.2587193347581, 325.021702725362], "content": "peese the function is harmonic: a sum of i hodonsorplise plus an antiliolomorphic plece:", "level": 1}, {"type": "text", "bbox": [27.095934107698895, 325.6666666668679, 138.95348822553743, 330.0000000002013], "content": "RI, is is casy to show that (F-) log(a - 6) o d\u00b0(a - 6). With these, we would find that p = 0. This argument contradicts that fp = 1, so the assumption that p is smooth does", "level": 1}, {"type": "text", "bbox": [26.401163403326393, 331.27212025329226, 138.25871752116493, 334.7445743243372], "content": "RI, is is casy to show that (F-) log(a - 6) o d\u00b0(a - 6). With these, we would find that p = 0. This argument contradicts that fp = 1, so the assumption that p is smooth does", "level": 1}, {"type": "text", "bbox": [27.095934107698895, 336.0000000122427, 138.95348822553743, 339.6666666789094], "content": "not work. This argument has boen given before in (5, 23), and we are presenting it bere for completeness", "level": 1}, {"type": "text", "bbox": [26.401163191155486, 340.9949916044331, 43.77034832095692, 344.467445675478], "content": "not work. This argument has boen given before in (5, 23), and we are presenting it bere for completeness", "level": 1}, {"type": "text", "bbox": [27.095935833103056, 346.55091820125887, 137.56395341122803, 348.6666666861883], "content": "siddhat this indicates is that the demity distribution of narticks le sinoular. Since tot", "level": 1}, {"type": "text", "bbox": [26.401173316814493, 351.9999999850441, 137.56395407853324, 354.33333331837747], "content": "9 = 0 the original probability distribution is spherically symmetric, it is natural to assume that the associated distribution is spherically summetric. The radius was computed in 118", "level": 1}, {"type": "text", "bbox": [26.401173316814493, 356.96828053382023, 137.56395407853324, 359.0517527857123], "content": "9 = 0 the original probability distribution is spherically symmetric, it is natural to assume that the associated distribution is spherically summetric. The radius was computed in 118", "level": 1}, {"type": "text", "bbox": [27.095943681066256, 361.6666666416275, 137.5639510866652, 363.9999999749609], "content": "Pitchas also been chocked macrically in 6. 7. A geocralization to all Calabi- Yau conc conforinal field tboory singularities for this result can be found in (23), where it was seen that", "level": 1}, {"type": "text", "bbox": [26.401162937253275, 365.9966611461136, 138.2587170550918, 369.4691152171585], "content": "Pitchas also been chocked macrically in 6. 7. A geocralization to all Calabi- Yau conc conforinal field tboory singularities for this result can be found in (23), where it was seen that", "level": 1}, {"type": "text", "bbox": [26.401166665838417, 370.85809680287485, 138.25872078367695, 374.33333337768363], "content": "this procedure matches vurious otber supergravity calculations in the abeence of symmetry.", "level": 1}, {"type": "text", "bbox": [27.095928261559823, 375.6666666154538, 138.25871919580462, 379.9999999487871], "content": "anger q > 0 but very small, we expect that the distribution does not change much, except in the region :' \u2265 0. Thus we expoct that the distribution gives a five dimensional manifold of", "level": 1}, {"type": "text", "bbox": [26.401168806551237, 380.58096827742037, 138.95348610798354, 384.74791309909597], "content": "anger q > 0 but very small, we expect that the distribution does not change much, except in the region :' \u2265 0. Thus we expoct that the distribution gives a five dimensional manifold of", "level": 1}, {"type": "text", "bbox": [26.401166665838417, 385.3333334078331, 138.25872078367695, 389.6666667411664], "content": "R\u00b0, possibly with a boundary, and that the region bear 2' = O has no density of eigenvalues present. After all, the effoctive potential repels all particles from the locus :' = 0. This", "level": 1}, {"type": "text", "bbox": [26.401162937253275, 390.30383982243984, 138.2587170550918, 394.47078464411544], "content": "R\u00b0, possibly with a boundary, and that the region bear 2' = O has no density of eigenvalues present. After all, the effoctive potential repels all particles from the locus :' = 0. This", "level": 1}, {"type": "text", "bbox": [210.4171795505282, -0.6308089336304192, 269.667123601635, 10.353680507287436], "content": "smolDocling", "level": 1}, {"type": "text", "bbox": [191.06104736975303, 27.000000068952545, 254.97965190263062, 30.666666735619174], "content": "Y/I\"Woekers engaged in essential or emergensy works.", "level": 1}, {"type": "text", "bbox": [192.45058253636216, 49.30884810023622, 243.1686052007502, 53.47579292191176], "content": "K/\"\"ential or emergency works\" inclades", "level": 1}, {"type": "text", "bbox": [198.70348810078428, 54.170283780874115, 271.65406996927385, 56.999999986470925], "content": "Greedons, maintenance, repairs, and asset replacement activities,", "level": 1}, {"type": "text", "bbox": [239.69476771998583, 60.42070119450443, 250.1162790521798, 64.58764601618009], "content": "2016(a).", "level": 1}, {"type": "text", "bbox": [226.4941893116771, 57.64273782300074, 291.10755702814845, 64.66666678316506], "content": "Repain 2 a. he lies Super it Quie)", "level": 1}, {"type": "text", "bbox": [264.0116281852332, 60.33333331766835, 280.68604631674356, 64.66666665100172], "content": "Supply (Water", "level": 1}, {"type": "text", "bbox": [280.6860470202226, 60.420701219522925, 291.8023253507076, 65.28213679182932], "content": "Quality", "level": 1}, {"type": "text", "bbox": [202.87209532184315, 63.89315523359278, 291.1075587821296, 67.3656093046377], "content": "Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the", "level": 1}, {"type": "text", "bbox": [369.61627919689363, 1.141631855716696e-08, 424.50290639415925, 9.028380723289729], "content": "Qwen2.5-VL", "level": 1}, {"type": "text", "bbox": [348.07849454955686, 27.666666637441494, 411.3023257263147, 29.999999970774752], "content": "21. (1) Workers encaned in cascatial or emergency works\u2014", "level": 1}, {"type": "text", "bbox": [348.7732569549668, 49.30884810023622, 399.4912796193548, 53.47579292191176], "content": "\"O) \"emal or emergency works\" includes-", "level": 1}, {"type": "text", "bbox": [355.7209365659078, 54.86477469074839, 427.9767450782776, 56.99999998676901], "content": "0) mspoctos, mamienance, repam, and asset replacement actioes and analysis of walcr", "level": 1}, {"type": "text", "bbox": [383.51162738534373, 58.99999999380077, 409.9127885791914, 61.11519185082079], "content": "0) mspoctos, mamienance, repam, and asset replacement actioes and analysis of walcr", "level": 1}, {"type": "text", "bbox": [410.60755836203515, 59.031719558444365, 418.9447674277903, 63.19866438011991], "content": "Watch", "level": 1}, {"type": "text", "bbox": [360.58430756971893, 63.89315523359278, 448.1249976738856, 67.3656093046377], "content": "Regulations 2016(b), the Private Water Supplies (Wales) Regulations 2017(c), or the", "level": 1}, {"type": "text", "bbox": [396.7122095804509, 60.42070119450443, 407.13372091264495, 64.58764601618009], "content": "2016(a).", "level": 1}, {"type": "text", "bbox": [421.0290699842453, 60.33333331303845, 434.9244188510259, 64.66666664637171], "content": "Supply (Wa", "level": 1}, {"type": "text", "bbox": [182.49, 68.42807499999992, 472.54291496399986, 259.73956592313607], "content": "Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. \"Wille C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on tbe suppost of p. We can prove that these equations lead to a singular support for p. We do this by contradiction.", "level": 1}, {"type": "text", "bbox": [183.4186056158078, 260.33333335582415, 214.68313961238982, 264.6666666891574], "content": "Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures. \"Wille C is a Lagrange multiplier enforcing the constraint (9). These equations are valid on tbe suppost of p. We can prove that these equations lead to a singular support for p. We do this by contradiction.", "level": 1}, {"type": "text", "bbox": [340.43604747627296, 261.0000000446156, 371.700581472855, 264.66666671128223], "content": "do this by contradiction.", "level": 1}, {"type": "picture", "bbox": [70.61808013916016, 265.51434326171875, 584.7910766601562, 692.1535034179688], "content": "", "level": 1}]}, {"page_number": 24, "width": 612.0, "height": 792.0, "elements": [{"type": "text", "bbox": [34.04360409124125, 24.271739241046816, 88.93023128850689, 27.739130393518394], "content": "DEVOTION TO SERVICE", "level": 1}, {"type": "text", "bbox": [27.095930233431805, 188.46972862127268, 38.90697556220784, 190.13465555237917], "content": "Net incorre heinrat let tormente", "level": 1}, {"type": "text", "bbox": [27.09593075855946, 195.46242171811286, 59.749996208592016, 196.94782600894723], "content": "Net incorre heinrat let tormente", "level": 1}, {"type": "text", "bbox": [27.790697373538563, 213.4436325338305, 38.906976975589345, 215.108559464937], "content": "Total astets", "level": 1}, {"type": "text", "bbox": [27.095930654399826, 219.10438419180014, 43.770348785910244, 222.76722344023437], "content": "Working capital", "level": 1}, {"type": "text", "bbox": [27.09593020404631, 226.07391306162663, 47.938952868434335, 229.5413042140981], "content": "Shareholders equity", "level": 1}, {"type": "text", "bbox": [27.040637310742923, 232.936225149081, 56.33097391962965, 236.5667939898883], "content": "Common shares outstanding", "level": 1}, {"type": "text", "bbox": [54.19186057351803, 257.28043475673144, 61.139535006908325, 260.0543478057324], "content": "$1171", "level": 1}, {"type": "text", "bbox": [67.39244164241632, 212.11169109495995, 79.20348824275811, 215.7745303433942], "content": "$ 875.228", "level": 1}, {"type": "text", "bbox": [67.39244164241632, 219.10438419180014, 79.20348824275811, 222.76722344023437], "content": "$ 265,375", "level": 1}, {"type": "text", "bbox": [67.39244164241632, 226.07391306162663, 79.20348824275811, 229.5413042140981], "content": "$ 436,616", "level": 1}, {"type": "text", "bbox": [71.56104685339383, 233.00869562629657, 79.20348828507515, 235.78260867529764], "content": "15.971", "level": 1}, {"type": "text", "bbox": [86.15116277002937, 187.2391305453947, 94.48837183578458, 190.70652169786615], "content": "38,104", "level": 1}, {"type": "text", "bbox": [83.37209308311397, 212.11169109495995, 95.18313968345576, 215.7745303433942], "content": "$ 1.197,188", "level": 1}, {"type": "text", "bbox": [83.37209317574964, 219.10438405613627, 95.18313977609144, 222.7672233045705], "content": "$ 295,648", "level": 1}, {"type": "text", "bbox": [83.37209317574964, 226.07391301760094, 95.18313977609144, 228.8478260666019], "content": "$ 451.169", "level": 1}, {"type": "text", "bbox": [87.54069801618454, 233.00869562629657, 95.18313944786587, 235.78260867529764], "content": "36.520", "level": 1}, {"type": "text", "bbox": [104.21511716907098, 24.96521742251946, 137.5639534320918, 29.968684741426614], "content": "Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [104.21511664836707, 46.46304345100282, 129.92151211548946, 50.62391302450442], "content": "Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [104.21511676506786, 51.9457201775017, 141.03778929110953, 56.27453019837856], "content": "employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [104.2151171134459, 56.86521743985645, 135.47965111002793, 61.71956511682845], "content": "employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [104.21511621083835, 62.26826725458284, 138.25872074371594, 67.26739131069621], "content": "families is a serious and dedicated", "level": 1}, {"type": "text", "bbox": [104.21511660613464, 67.92901880153897, 129.22674380340027, 72.25782882241583], "content": "families is a serious and dedicated", "level": 1}, {"type": "text", "bbox": [104.21511699002617, 73.50869557868407, 128.53197591743503, 77.66956515218556], "content": "commitment.", "level": 1}, {"type": "text", "bbox": [27.790698682705, 135.85803759335045, 69.47674401148105, 140.1868476142273], "content": "FINANCIAL HIGHLIGHTS", "level": 1}, {"type": "text", "bbox": [27.790699408846535, 142.16304351814438, 138.95349034309132, 147.17954078421303], "content": "VISA's ongoing commitment to investing in core products and focus on managing costs helped generate a strong finish in 2011.", "level": 1}, {"type": "text", "bbox": [27.790699135047173, 147.71086962821687, 143.81686287202635, 152.56521730518887], "content": "VISA's ongoing commitment to investing in core products and focus on managing costs helped generate a strong finish in 2011.", "level": 1}, {"type": "text", "bbox": [120.1947682135765, 169.1565762223521, 145.2063954108421, 172.81941547078634], "content": "CROCURR TUAL SALES", "level": 1}, {"type": "text", "bbox": [120.19476749110572, 172.67608705846533, 149.37499922124894, 176.14926938677513], "content": "BY CORE PRODUCT GROUP", "level": 1}, {"type": "text", "bbox": [122.97383798101833, 214.28478258281825, 147.98546517828396, 217.10647184011157], "content": "\u00a9 Fund Gin A Fume Detectior", "level": 1}, {"type": "text", "bbox": [122.97383785239903, 217.75217395430775, 143.81686051678705, 221.2195651067792], "content": "\u00b7 Poutis Car Outection", "level": 1}, {"type": "text", "bbox": [122.27906956221783, 223.99347827901795, 138.25872069543723, 227.4608694314894], "content": "@ Other hodets", "level": 1}, {"type": "text", "bbox": [102.1308141180914, 187.2391304529309, 110.4680231838466, 190.70652160540237], "content": "60152", "level": 1}, {"type": "text", "bbox": [104.90988390575085, 194.17391306162654, 110.46802307099335, 197.6413042140981], "content": "1.91", "level": 1}, {"type": "text", "bbox": [98.65697651046685, 213.59130435106852, 109.77325611251763, 214.97826087556905], "content": "S1015092", "level": 1}, {"type": "text", "bbox": [98.65697680404422, 219.10438419180014, 110.46802340438599, 222.76722344023437], "content": "$ 287,079", "level": 1}, {"type": "text", "bbox": [98.65697680404422, 226.07391306162663, 110.46802340438599, 229.5413042140981], "content": "$ 433,666", "level": 1}, {"type": "text", "bbox": [102.13081421072702, 233.0086956934382, 110.46802327648226, 236.47608684590966], "content": "36,493", "level": 1}, {"type": "text", "bbox": [191.75581526426072, 23.578260837683956, 245.94767419166956, 27.739130411185556], "content": "DEVOTION TO SERVICE", "level": 1}, {"type": "text", "bbox": [259.14825587375134, 24.965217474530277, 294.58139440321094, 29.968684793437433], "content": "\"Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [261.2325584017222, 46.46304345678186, 286.9389538688446, 50.62391303028335], "content": "\"Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [259.1482568708299, 51.94572028427069, 298.74999993316715, 56.27453030514744], "content": "-employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [261.23255840941255, 56.86521754676755, 292.4970924059946, 61.719565223739664], "content": "-employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [261.2325586242225, 62.26826723642034, 295.27616315710014, 66.5970772572972], "content": "families is a serious", "level": 1}, {"type": "text", "bbox": [418.94476865554003, 24.965217431187966, 451.5988366487041, 29.96868475009512], "content": "Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [418.24999951241756, 46.46304352035065, 443.95639497953994, 50.62391309385225], "content": "Giving back to the and where our", "level": 1}, {"type": "text", "bbox": [418.2499999069926, 51.94572018905967, 455.76744070289107, 56.27453020993653], "content": "employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [418.25000026987755, 56.86521754676755, 449.5145342664596, 61.719565223739664], "content": "employees live, work, and care for their", "level": 1}, {"type": "text", "bbox": [418.24999993176857, 62.26826725458284, 452.2936044646462, 67.26739131069621], "content": "families is a serious", "level": 1}, {"type": "text", "bbox": [182.49, 68.42807499999992, 472.54291496399986, 256.74238820000005], "content": "Table 7: Visualizations of layout output from SmolDocling and QwenVL2.5 compared to the DocLayNet ground truth. Examples are chosen to be representative of different layout styles and features. The prediction results however do not represent a generalizable measure of the model's performance on inputs with similar features. (1) Multi-column pages are handled by SmolDocling and Qwen2.5-VL, with some recall errors in the latter. (2) A manual page with terminal output shows poor bounding box recall on SmolDocling and label confusion in Qwen2.5-VL. (3) Lists with nesting are handled well in SmolDocling but confuse Qwen2.5-VL. (4) Both SmolDocling and Qwen2.5-VL reconstruct equations well, however with different annotation conventions on including or excluding the equation index. (5) On a portrait page with colorful elements and gradient background, SmolDocling creates less accurate bounding-boxes and Qwen2.5-VL suffers low recall. (6) On a report page with tables and diagrams, SmolDocling output exhibits some repetition loop, fabricating non-existent text cells (bottom left), while Qwen2.5-VL is confused between tables and pictures.", "level": 1}, {"type": "text", "bbox": [261.23255833246395, 258.66739134561203, 267.48546513178036, 262.1347824980835], "content": "340.9", "level": 1}, {"type": "text", "bbox": [348.77325654852, 24.271739204121445, 402.9651154759288, 27.73913035659291], "content": "DEVOTION TO SERVICE", "level": 1}, {"type": "text", "bbox": [441.17732585885165, 262.13478264996, 446.7354650240941, 265.6021738024315], "content": "22%", "level": 1}, {"type": "picture", "bbox": [70.86924743652344, 266.0121765136719, 585.3001098632812, 597.5089721679688], "content": "", "level": 1}]}]


