xiaoxue_admin
/
dolphin
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

同步完整源码 - 2026-05-24

This commit is contained in:
xiaoxue 2026-05-24 23:28:07 +08:00
commit 32e645b6af
23 changed files with 20398 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

154
.gitignore vendored Normal file
View File

@ -0,0 +1,154 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
coverage.xml
*.mo
*.pot
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
.idea/
*.iml
# VS Code
.vscode/
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
# macOS
.DS_Store
# Windows
Thumbs.db
ehthumbs.db
Desktop.ini
fusion_result.json
kernel_meta/

34
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,34 @@
repos:
# 1. isort - 自动排序 Python imports
- repo: https://github.com/pycqa/isort
rev: 6.0.1 # 使用固定版本号
hooks:
- id: isort
name: isort (python)
args: [--profile=black] # 与 Black 兼容的配置
language: python
# 2. Black - 自动格式化 Python 代码
- repo: https://github.com/psf/black
rev: 25.1.0 # 使用固定版本号
hooks:
- id: black
language: python
# 3. flake8 - Python 静态检查
- repo: https://github.com/pycqa/flake8
rev: 7.2.0
hooks:
- id: flake8
args: [--max-line-length=120, --ignore=E203] # 设置行长度为 120
additional_dependencies: [flake8-bugbear==24.12.12] # 可选：增强检查
# 4. pre-commit-hooks - 通用 Git 钩子
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace # 删除行尾空格
- id: end-of-file-fixer # 确保文件以换行符结束
- id: check-yaml # 验证 YAML 文件语法
- id: check-added-large-files # 阻止大文件提交
args: ["--maxkb=512"]

54
LICENSE Normal file
View File

@ -0,0 +1,54 @@
Qwen RESEARCH LICENSE AGREEMENT
Qwen RESEARCH LICENSE AGREEMENT Release Date: September 19, 2024
By clicking to agree or by using or distributing any portion or element of the Qwen Materials, you will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
1. Definitions
a. This Qwen RESEARCH LICENSE AGREEMENT (this "Agreement") shall mean the terms and conditions for use, reproduction, distribution and modification of the Materials as defined by this Agreement.
b. "We" (or "Us") shall mean Alibaba Cloud.
c. "You" (or "Your") shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Materials for any purpose and in any field of use.
d. "Third Parties" shall mean individuals or legal entities that are not under common control with us or you.
e. "Qwen" shall mean the large language models, and software and algorithms, consisting of trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing distributed by us.
f. "Materials" shall mean, collectively, Alibaba Cloud's proprietary Qwen and Documentation (and any portion thereof) made available under this Agreement.
g. "Source" form shall mean the preferred form for making modifications, including but not limited to model source code, documentation source, and configuration files.
h. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
i. "Non-Commercial" shall mean for research or evaluation purposes only.
2. Grant of Rights
a. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Alibaba Cloud's intellectual property or other rights owned by us embodied in the Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Materials FOR NON-COMMERCIAL PURPOSES ONLY.
b. If you are commercially using the Materials, you shall request a license from us.
3. Redistribution
You may distribute copies or make the Materials, or derivative works thereof, available as part of a product or service that contains any of them, with or without modifications, and in Source or Object form, provided that you meet the following conditions:
a. You shall give any other recipients of the Materials or derivative works a copy of this Agreement;
b. You shall cause any modified files to carry prominent notices stating that you changed the files;
c. You shall retain in all copies of the Materials that you distribute the following attribution notices within a "Notice" text file distributed as a part of such copies: "Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) Alibaba Cloud. All Rights Reserved."; and
d. You may add your own copyright statement to your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of your modifications, or for any such derivative works as a whole, provided your use, reproduction, and distribution of the work otherwise complies with the terms and conditions of this Agreement.
4. Rules of use
a. The Materials may be subject to export controls or restrictions in China, the United States or other countries or regions. You shall comply with applicable laws and regulations in your use of the Materials.
b. If you use the Materials or any outputs or results therefrom to create, train, fine-tune, or improve an AI model that is distributed or made available, you shall prominently display “Built with Qwen” or “Improved using Qwen” in the related product documentation.
5. Intellectual Property
a. We retain ownership of all intellectual property rights in and to the Materials and derivatives made by or for us. Conditioned upon compliance with the terms and conditions of this Agreement, with respect to any derivative works and modifications of the Materials that are made by you, you are and will be the owner of such derivative works and modifications.
b. No trademark license is granted to use the trade names, trademarks, service marks, or product names of us, except as required to fulfill notice requirements under this Agreement or as required for reasonable and customary use in describing and redistributing the Materials.
c. If you commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against us or any entity alleging that the Materials or any output therefrom, or any part of the foregoing, infringe any intellectual property or other right owned or licensable by you, then all licenses granted to you under this Agreement shall terminate as of the date such lawsuit or other proceeding is commenced or brought.
6. Disclaimer of Warranty and Limitation of Liability
a. We are not obligated to support, update, provide training for, or develop any further version of the Qwen Materials or to grant any license thereto.
b. THE MATERIALS ARE PROVIDED "AS IS" WITHOUT ANY EXPRESS OR IMPLIED WARRANTY OF ANY KIND INCLUDING WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE. WE MAKE NO WARRANTY AND ASSUME NO RESPONSIBILITY FOR THE SAFETY OR STABILITY OF THE MATERIALS AND ANY OUTPUT THEREFROM.
c. IN NO EVENT SHALL WE BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE MATERIALS OR ANY OUTPUT OF IT, NO MATTER HOW IT’S CAUSED.
d. You will defend, indemnify and hold harmless us from and against any claim by any third party arising out of or related to your use or distribution of the Materials.
7. Survival and Termination.
a. The term of this Agreement shall commence upon your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
b. We may terminate this Agreement if you breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, you must delete and cease use of the Materials. Sections 6 and 8 shall survive the termination of this Agreement.
8. Governing Law and Jurisdiction.
a. This Agreement and any dispute arising out of or relating to it will be governed by the laws of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
b. The People's Courts in Hangzhou City shall have exclusive jurisdiction over any dispute arising out of this Agreement.
9. Other Terms and Conditions.
a. Any arrangements, understandings, or agreements regarding the Material not stated herein are separate from and independent of the terms and conditions of this Agreement. You shall request a separate license from us, if you use the Materials in ways not expressly agreed to in this Agreement.
b. We shall not be bound by any additional or different terms or conditions communicated by you unless expressly agreed.

226
README.md Normal file
View File

@ -0,0 +1,226 @@
<div align="center">
<img src="./assets/dolphin.png" width="300">
</div>
<div align="center">
<a href="https://arxiv.org/abs/2505.14059">
<img src="https://img.shields.io/badge/Paper-arXiv-red">
</a>
<a href="https://huggingface.co/ByteDance/Dolphin-v2">
<img src="https://img.shields.io/badge/HuggingFace-Dolphin-yellow">
</a>
<a href="https://github.com/bytedance/Dolphin">
<img src="https://img.shields.io/badge/Code-Github-green">
</a>
<a href="https://opensource.org/licenses/MIT">
<img src="https://img.shields.io/badge/License-MIT-lightgray">
</a>
<br>
</div>
<br>
<div align="center">
<img src="./assets/demo.gif" width="800">
</div>
# Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting
Dolphin-v2 is an enhanced universal document parsing model that substantially improves upon the original Dolphin. It seamlessly handles any document type—whether digital-born or photographed—through a document-type-aware two-stage architecture with scalable anchor prompting.
## 📑 Overview
Document image parsing is challenging due to diverse document types and complexly intertwined elements such as text paragraphs, figures, formulas, tables, and code blocks. Dolphin-v2 addresses these challenges through a document-type-aware two-stage approach:
1. **🔍 Stage 1**: Document type classification (digital vs. photographed) + layout analysis with reading order prediction
2. **🧩 Stage 2**: Hybrid parsing strategy - holistic parsing for photographed documents, parallel element-wise parsing for digital documents
<div align="center">
<img src="./assets/framework.png" width="680">
</div>
Dolphin achieves promising performance across diverse page-level and element-level parsing tasks while ensuring superior efficiency through its lightweight architecture and parallel parsing mechanism.
<!-- ## 🚀 Demo
Try our demo on [Demo-Dolphin](https://huggingface.co/spaces/ByteDance/Dolphin). -->
## 📅 Changelog
- 🔥 **2025.12.12** Released *Dolphin-v2* model. Upgraded to 3B parameters with 21-element detection, attribute field extraction, dedicated formula/code parsing, and robust photographed document parsing. (Dolphin-1.5 moved to [v1.5 branch](https://github.com/bytedance/Dolphin/tree/v1.5))
- 🔥 **2025.10.16** Released *Dolphin-1.5* model. While maintaining the lightweight 0.3B architecture, this version achieves significant parsing improvements. (Dolphin 1.0 moved to [v1.0 branch](https://github.com/bytedance/Dolphin/tree/v1.0))
- 🔥 **2025.07.10** Released the *Fox-Page Benchmark*, a manually refined subset of the original [Fox dataset](https://github.com/ucaslcl/Fox). Download via: [Baidu Yun](https://pan.baidu.com/share/init?surl=t746ULp6iU5bUraVrPlMSw&pwd=fox1) | [Google Drive](https://drive.google.com/file/d/1yZQZqI34QCqvhB4Tmdl3X_XEvYvQyP0q/view?usp=sharing).
- 🔥 **2025.06.30** Added [TensorRT-LLM support](https://github.com/bytedance/Dolphin/blob/master/deployment/tensorrt_llm/ReadMe.md) for accelerated inference！
- 🔥 **2025.06.27** Added [vLLM support](https://github.com/bytedance/Dolphin/blob/master/deployment/vllm/ReadMe.md) for accelerated inference！
- 🔥 **2025.06.13** Added multi-page PDF document parsing capability.
- 🔥 **2025.05.21** Our demo is released at [link](http://115.190.42.15:8888/dolphin/). Check it out!
- 🔥 **2025.05.20** The pretrained model and inference code of Dolphin are released.
- 🔥 **2025.05.16** Our paper has been accepted by ACL 2025. Paper link: [arXiv](https://arxiv.org/abs/2505.14059).
## 📈 Performance
<table style="width:90%; border-collapse: collapse; text-align: center;">
<caption>Comprehensive evaluation of document parsing on OmniDocBench (v1.5)</caption>
<thead>
<tr>
<th style="text-align: center !important;">Model</th>
<th style="text-align: center !important;">Size</th>
<th style="text-align: center !important;">Overall&#x2191;</th>
<th style="text-align: center !important;">Text<sup>Edit</sup>&#x2193;</th>
<th style="text-align: center !important;">Formula<sup>CDM</sup>&#x2191;</th>
<th style="text-align: center !important;">Table<sup>TEDS</sup>&#x2191;</th>
<th style="text-align: center !important;">Table<sup>TEDS-S</sup>&#x2191;</th>
<th style="text-align: center !important;">Read Order<sup>Edit</sup>&#x2193;</th>
</tr>
</thead>
<tbody>
<tr>
<td>Dolphin</td>
<td>0.3B</td>
<td>74.67</td>
<td>0.125</td>
<td>67.85</td>
<td>68.70</td>
<td>77.77</td>
<td>0.124</td>
</tr>
<tr>
<td>Dolphin-1.5</td>
<td>0.3B</td>
<td>85.06</td>
<td>0.085</td>
<td>79.44</td>
<td>84.25</td>
<td>88.06</td>
<td>0.071</td>
</tr>
<tr>
<td>Dolphin-v2</td>
<td>3B</td>
<td><strong>89.78</strong></td>
<td><strong>0.054</strong></td>
<td><strong>87.63</strong></td>
<td><strong>87.02</strong></td>
<td><strong>90.48</strong></td>
<td><strong>0.054</strong></td>
</tr>
</tbody>
</table>
## 🛠️ Installation
1. Clone the repository:
```bash
git clone https://github.com/ByteDance/Dolphin.git
cd Dolphin
```
2. Install the dependencies:
```bash
pip install -r requirements.txt
```
3. Download the pre-trained models of *Dolphin-v2*:
Visit our Huggingface [model card](https://huggingface.co/ByteDance/Dolphin-v2), or download model by:
```bash
# Download the model from Hugging Face Hub
git lfs install
git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model
# Or use the Hugging Face CLI
pip install huggingface_hub
huggingface-cli download ByteDance/Dolphin-v2 --local-dir ./hf_model
```
## ⚡ Inference
Dolphin provides two inference frameworks with support for two parsing granularities:
- **Page-level Parsing**: Parse the entire document page into a structured JSON and Markdown format
- **Element-level Parsing**: Parse individual document elements (text, table, formula)
### 📄 Page-level Parsing
```bash
# Process a single document image
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_1.png
# Process a single document pdf
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_6.pdf
# Process all documents in a directory
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs
# Process with custom batch size for parallel element decoding
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs \
--max_batch_size 8
```
### 🧩 Element-level Parsing
````bash
# Process element images (specify element_type: table, formula, text, or code)
python demo_element.py --model_path ./hf_model --save_dir ./results \
--input_path \
--element_type [table|formula|text|code]
````
### 🎨 Layout Parsing
````bash
# Process a single document image
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_1.png \
# Process a single PDF document
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_6.pdf \
# Process all documents in a directory
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs
````
## 🌟 Key Features
- 🔄 Two-stage analyze-then-parse approach based on a single VLM
- 📊 Promising performance on document parsing tasks
- 🔍 Natural reading order element sequence generation
- 🧩 Heterogeneous anchor prompting for different document elements
- ⏱️ Efficient parallel parsing mechanism
- 🤗 Support for Hugging Face Transformers for easier integration
## 📮 Notice
**Call for Bad Cases:** If you have encountered any cases where the model performs poorly, we would greatly appreciate it if you could share them in the issue. We are continuously working to optimize and improve the model.
## 💖 Acknowledgement
We would like to acknowledge the following open-source projects that provided inspiration and reference for this work:
- [OmniDocBench](https://github.com/opendatalab/OmniDocBench)
- [Donut](https://github.com/clovaai/donut/)
- [Nougat](https://github.com/facebookresearch/nougat)
- [GOT](https://github.com/Ucas-HaoranWei/GOT-OCR2.0)
- [MinerU](https://github.com/opendatalab/MinerU/tree/master)
- [Swin](https://github.com/microsoft/Swin-Transformer)
- [Hugging Face Transformers](https://github.com/huggingface/transformers)
## 📝 Citation
If you find this code useful for your research, please use the following BibTeX entry.
```bibtex
@article{feng2025dolphin,
title={Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting},
author={Feng, Hao and Wei, Shu and Fei, Xiang and Shi, Wei and Han, Yingdong and Liao, Lei and Lu, Jinghui and Wu, Binghong and Liu, Qi and Lin, Chunhui and others},
journal={arXiv preprint arXiv:2505.14059},
year={2025}
}
```
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/Dolphin&type=Date)](https://www.star-history.com/#bytedance/Dolphin&Date)

223
README_CN.md Normal file
View File

@ -0,0 +1,223 @@
<div align="center">
<img src="./assets/dolphin.png" width="300">
</div>
<div align="center">
<a href="https://arxiv.org/abs/2505.14059">
<img src="https://img.shields.io/badge/论文-arXiv-red">
</a>
<a href="https://huggingface.co/ByteDance/Dolphin-v2">
<img src="https://img.shields.io/badge/HuggingFace-Dolphin-yellow">
</a>
<a href="https://github.com/bytedance/Dolphin">
<img src="https://img.shields.io/badge/代码-Github-green">
</a>
<a href="https://opensource.org/licenses/MIT">
<img src="https://img.shields.io/badge/许可证-MIT-lightgray">
</a>
<br>
</div>
<br>
<div align="center">
<img src="./assets/demo.gif" width="800">
</div>
# Dolphin: 基于异构锚点提示的文档图像解析
Dolphin（**Do**cument Image **P**arsing via **H**eterogeneous Anchor Prompt**in**g）是一个创新的多模态文档图像解析模型（**0.3B**），采用"分析-解析"的两阶段范式。本仓库包含Dolphin的演示代码和预训练模型。
## 📑 概述
由于文档图像中文本段落、图表、公式和表格等元素的复杂交织，文档图像解析具有挑战性。Dolphin通过两阶段方法解决这些挑战：
1. **🔍 第一阶段**：通过按自然阅读顺序生成元素序列进行全面的页面级布局分析
2. **🧩 第二阶段**：使用异构锚点和任务特定提示高效并行解析文档元素
<div align="center">
<img src="./assets/framework.png" width="680">
</div>
Dolphin在多样化的页面级和元素级解析任务中取得了优异的性能，同时通过其轻量级架构和并行解析机制确保了卓越的效率。
## 📅 更新日志
- 🔥 **2025.12.12** *Dolphin-v2* 开源！支持 21 类元素检测、属性字段提取、代码专用解析，以及拍照文档解析。（原1.5版本已迁移至[v1.5分支](https://github.com/bytedance/Dolphin/tree/v1.5)）
- 🔥 **2025.10.16** *Dolphin-1.5* 开源！在保持轻量级0.3B架构的同时，该版本实现了显著的解析性能提升。（原1.0版本已迁移至[v1.0分支](https://github.com/bytedance/Dolphin/tree/v1.0)）
- 🔥 **2025.07.10** *Fox-Page* 基准测试开源。这是原始 [Fox 数据集](https://github.com/ucaslcl/Fox) 人工矫正标注后的版本。下载地址：[百度网盘](https://pan.baidu.com/share/init?surl=t746ULp6iU5bUraVrPlMSw&pwd=fox1) | [Google Drive](https://drive.google.com/file/d/1yZQZqI34QCqvhB4Tmdl3X_XEvYvQyP0q/view?usp=sharing)。
- 🔥 **2025.06.30** 新增[TensorRT-LLM](https://github.com/bytedance/Dolphin/blob/master/deployment/tensorrt_llm/ReadMe.md)支持，提升推理速度！
- 🔥 **2025.06.27** 新增[vLLM](https://github.com/bytedance/Dolphin/blob/master/deployment/vllm/ReadMe.md)支持，提升推理速度！
- 🔥 **2025.06.13** 新增多页PDF文档解析功能。
- 🔥 **2025.05.21** 我们的演示已在 [链接](http://115.190.42.15:8888/dolphin/) 发布。快来体验吧！
- 🔥 **2025.05.20** Dolphin的预训练模型和推理代码已发布。
- 🔥 **2025.05.16** 我们的论文已被ACL 2025接收。论文链接：[arXiv](https://arxiv.org/abs/2505.14059)。
## 📈 性能表现
<table style="width:90%; border-collapse: collapse; text-align: center;">
<caption>OmniDocBench (v1.5) 测试基准上评估结果</caption>
<thead>
<tr>
<th style="text-align: center !important;">模型</th>
<th style="text-align: center !important;">参数</th>
<th style="text-align: center !important;">总体&#x2191;</th>
<th style="text-align: center !important;">文本<sup>Edit</sup>&#x2193;</th>
<th style="text-align: center !important;">公式<sup>CDM</sup>&#x2191;</th>
<th style="text-align: center !important;">表格<sup>TEDS</sup>&#x2191;</th>
<th style="text-align: center !important;">表格<sup>TEDS-S</sup>&#x2191;</th>
<th style="text-align: center !important;">阅读顺序<sup>Edit</sup>&#x2193;</th>
</tr>
</thead>
<tr>
<td>Dolphin</td>
<td>0.3B</td>
<td>74.67</td>
<td>0.125</td>
<td>67.85</td>
<td>68.70</td>
<td>77.77</td>
<td>0.124</td>
</tr>
<tr>
<td>Dolphin-1.5</td>
<td>0.3B</td>
<td>85.06</td>
<td>0.085</td>
<td>79.44</td>
<td>84.25</td>
<td>88.06</td>
<td>0.071</td>
</tr>
<tr>
<td>Dolphin-v2</td>
<td>0.3B</td>
<td><strong>89.78</strong></td>
<td><strong>0.054</strong></td>
<td><strong>87.63</strong></td>
<td><strong>87.02</strong></td>
<td><strong>90.48</strong></td>
<td><strong>0.054</strong></td>
</tr>
</tbody>
</table>
## 🛠️ 安装
1. 克隆仓库：
```bash
git clone https://github.com/ByteDance/Dolphin.git
cd Dolphin
```
2. 安装依赖：
```bash
pip install -r requirements.txt
```
3. 使用以下选项之一下载 *Dolphin-v2* 的预训练模型：
访问我们的Huggingface [模型卡片](https://huggingface.co/ByteDance/Dolphin-v2)，或通过以下方式下载模型：
```bash
# 从Hugging Face Hub下载模型
git lfs install
git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model
# 或使用Hugging Face CLI
pip install huggingface_hub
huggingface-cli download ByteDance/Dolphin-v2 --local-dir ./hf_model
```
## ⚡ 推理
Dolphin提供两个推理框架，支持两种解析粒度：
- **页面级解析**：将整个文档页面解析为结构化的JSON和Markdown格式
- **元素级解析**：解析单个文档元素（文本、表格、公式）
### 📄 页面级解析
```bash
# 处理单个文档图像
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_1.png
# 处理单个文档PDF
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_6.pdf
# 处理目录中的所有文档
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs
# 使用自定义批次大小进行并行元素解码
python demo_page.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs \
--max_batch_size 8
```
### 🧩 元素级解析
````bash
# 解析块图像 (支持块图像类型: table, formula, text, or code)
python demo_element.py --model_path ./hf_model --save_dir ./results \
--input_path \
--element_type [table|formula|text|code]
````
### 🎨 元素定位及阅读顺序解析
````bash
# 处理单个文档图像
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_1.png \
# 处理单个文档PDF
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs/page_6.pdf \
# 处理目录中的所有文档
python demo_layout.py --model_path ./hf_model --save_dir ./results \
--input_path ./demo/page_imgs
````
## 🌟 主要特性
- 🔄 基于单一VLM的两阶段分析-解析方法
- 📊 在文档解析任务上的优异性能
- 🔍 自然阅读顺序元素序列生成
- 🧩 针对不同文档元素的异构锚点提示
- ⏱️ 高效的并行解析机制
- 🤗 支持Hugging Face Transformers，便于集成
## 📮 通知
**征集不良案例：** 如果您遇到模型表现不佳的案例，我们非常欢迎您在issue中分享。我们正在持续优化和改进模型。
## 💖 致谢
我们要感谢以下开源项目为本工作提供的灵感和参考：
- [OmniDocBench](https://github.com/opendatalab/OmniDocBench)
- [Donut](https://github.com/clovaai/donut/)
- [Nougat](https://github.com/facebookresearch/nougat)
- [GOT](https://github.com/Ucas-HaoranWei/GOT-OCR2.0)
- [MinerU](https://github.com/opendatalab/MinerU/tree/master)
- [Swin](https://github.com/microsoft/Swin-Transformer)
- [Hugging Face Transformers](https://github.com/huggingface/transformers)
## 📝 引用
如果您在研究中发现此代码有用，请使用以下BibTeX条目。
```bibtex
@article{feng2025dolphin,
title={Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting},
author={Feng, Hao and Wei, Shu and Fei, Xiang and Shi, Wei and Han, Yingdong and Liao, Lei and Lu, Jinghui and Wu, Binghong and Liu, Qi and Lin, Chunhui and others},
journal={arXiv preprint arXiv:2505.14059},
year={2025}
}
```
## 星标历史
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/Dolphin&type=Date)](https://www.star-history.com/#bytedance/Dolphin&Date)

BIN
assets/demo.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 MiB

BIN
assets/dolphin.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 81 KiB

BIN
assets/framework.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.4 MiB

BIN
demo/element_imgs/block_formula.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
demo/element_imgs/code.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 383 KiB

BIN
demo/element_imgs/line_formula.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 54 KiB

BIN
demo/element_imgs/para_1.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
demo/element_imgs/para_2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB

BIN
demo/element_imgs/para_3.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 82 KiB

BIN
demo/element_imgs/table.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 167 KiB

BIN
demo/page_imgs/page_0.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 920 KiB

BIN
demo/page_imgs/page_1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 207 KiB

BIN
demo/page_imgs/page_2.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
demo/page_imgs/page_3.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 439 KiB

BIN
demo/page_imgs/page_4.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 363 KiB

BIN
demo/page_imgs/page_5.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 626 KiB

654
demo/page_imgs/page_6.pdf Normal file
View File

@ -0,0 +1,654 @@
Deep Residual Learning for Image Recognition
Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
Microsoft Research
{kahe, v-xiangz, v-shren, jiansun}@microsoft.com
Abstract 20 20
Deeper neural networks are more difficult to train. We training error (%) 56-layer
present a residual learning framework to ease the training test error (%)
of networks that are substantially deeper than those used 10 10 20-layer
previously. We explicitly reformulate the layers as learn-
ing residual functions with reference to the layer inputs, in- 56-layer
stead of learning unreferenced functions. We provide com-
prehensive empirical evidence showing that these residual 20-layer
networks are easier to optimize, and can gain accuracy from
considerably increased depth. On the ImageNet dataset we 00 1 2 3 4 5 6 00 1 2 3 4 5 6
evaluate residual nets with a depth of up to 152 layers—8×
deeper than VGG nets [40] but still having lower complex- iter. (1e4) iter. (1e4)
ity. An ensemble of these residual nets achieves 3.57% error
on the ImageNet test set. This result won the 1st place on the Figure 1. Training error (left) and test error (right) on CIFAR-10
ILSVRC 2015 classification task. We also present analysis
on CIFAR-10 with 100 and 1000 layers. with 20-layer and 56-layer “plain” networks. The deeper network
The depth of representations is of central importance has higher training error, and thus test error. Similar phenomena
for many visual recognition tasks. Solely due to our ex-
tremely deep representations, we obtain a 28% relative im- on ImageNet is presented in Fig. 4.
provement on the COCO object detection dataset. Deep
residual nets are foundations of our submissions to ILSVRC greatly benefited from very deep models.
& COCO 2015 competitions1, where we also won the 1st
places on the tasks of ImageNet detection, ImageNet local- Driven by the significance of depth, a question arises: Is
ization, COCO detection, and COCO segmentation. learning better networks as easy as stacking more layers?
An obstacle to answering this question was the notorious
1. Introduction problem of vanishing/exploding gradients [14, 1, 8], which
hamper convergence from the beginning. This problem,
Deep convolutional neural networks [22, 21] have led however, has been largely addressed by normalized initial-
to a series of breakthroughs for image classification [21, ization [23, 8, 36, 12] and intermediate normalization layers
49, 39]. Deep networks naturally integrate low/mid/high- [16], which enable networks with tens of layers to start con-
level features [49] and classifiers in an end-to-end multi- verging for stochastic gradient descent (SGD) with back-
layer fashion, and the “levels” of features can be enriched propagation [22].
by the number of stacked layers (depth). Recent evidence
[40, 43] reveals that network depth is of crucial importance, When deeper networks are able to start converging, a
and the leading results [40, 43, 12, 16] on the challenging degradation problem has been exposed: with the network
ImageNet dataset [35] all exploit “very deep” [40] models, depth increasing, accuracy gets saturated (which might be
with a depth of sixteen [40] to thirty [16]. Many other non- unsurprising) and then degrades rapidly. Unexpectedly,
trivial visual recognition tasks [7, 11, 6, 32, 27] have also such degradation is not caused by overfitting, and adding
more layers to a suitably deep model leads to higher train-
1http://image-net.org/challenges/LSVRC/2015/ and ing error, as reported in [10, 41] and thoroughly verified by
http://mscoco.org/dataset/#detections-challenge2015. our experiments. Fig. 1 shows a typical example.
The degradation (of training accuracy) indicates that not
all systems are similarly easy to optimize. Let us consider a
shallower architecture and its deeper counterpart that adds
more layers onto it. There exists a solution by construction
to the deeper model: the added layers are identity mapping,
and the other layers are copied from the learned shallower
model. The existence of this constructed solution indicates
that a deeper model should produce no higher training error
than its shallower counterpart. But experiments show that
our current solvers on hand are unable to find solutions that
1770
x ImageNet test set, and won the 1st place in the ILSVRC
2015 classification competition. The extremely deep rep-
F(x) weight layer x resentations also have excellent generalization performance
relu on other recognition tasks, and lead us to further win the
identity 1st places on: ImageNet detection, ImageNet localization,
weight layer COCO detection, and COCO segmentation in ILSVRC &
COCO 2015 competitions. This strong evidence shows that
F(x)+x relu the residual learning principle is generic, and we expect that
it is applicable in other vision and non-vision problems.
Figure 2. Residual learning: a building block.
2. Related Work
are comparably good or better than the constructed solution
(or unable to do so in feasible time). Residual Representations. In image recognition, VLAD
[18] is a representation that encodes by the residual vectors
In this paper, we address the degradation problem by with respect to a dictionary, and Fisher Vector [30] can be
introducing a deep residual learning framework. In- formulated as a probabilistic version [18] of VLAD. Both
stead of hoping each few stacked layers directly fit a of them are powerful shallow representations for image re-
desired underlying mapping, we explicitly let these lay- trieval and classification [4, 47]. For vector quantization,
ers fit a residual mapping. Formally, denoting the desired encoding residual vectors [17] is shown to be more effec-
underlying mapping as H(x), we let the stacked nonlinear tive than encoding original vectors.
layers fit another mapping of F(x) := H(x) − x. The orig-
inal mapping is recast into F(x)+x. We hypothesize that it In low-level vision and computer graphics, for solv-
is easier to optimize the residual mapping than to optimize ing Partial Differential Equations (PDEs), the widely used
the original, unreferenced mapping. To the extreme, if an Multigrid method [3] reformulates the system as subprob-
identity mapping were optimal, it would be easier to push lems at multiple scales, where each subproblem is respon-
the residual to zero than to fit an identity mapping by a stack sible for the residual solution between a coarser and a finer
of nonlinear layers. scale. An alternative to Multigrid is hierarchical basis pre-
conditioning [44, 45], which relies on variables that repre-
The formulation of F(x) + x can be realized by feedfor- sent residual vectors between two scales. It has been shown
ward neural networks with “shortcut connections” (Fig. 2). [3, 44, 45] that these solvers converge much faster than stan-
Shortcut connections [2, 33, 48] are those skipping one or dard solvers that are unaware of the residual nature of the
more layers. In our case, the shortcut connections simply solutions. These methods suggest that a good reformulation
perform identity mapping, and their outputs are added to or preconditioning can simplify the optimization.
the outputs of the stacked layers (Fig. 2). Identity short-
cut connections add neither extra parameter nor computa- Shortcut Connections. Practices and theories that lead to
tional complexity. The entire network can still be trained shortcut connections [2, 33, 48] have been studied for a long
end-to-end by SGD with backpropagation, and can be eas- time. An early practice of training multi-layer perceptrons
ily implemented using common libraries (e.g., Caffe [19]) (MLPs) is to add a linear layer connected from the network
without modifying the solvers. input to the output [33, 48]. In [43, 24], a few interme-
diate layers are directly connected to auxiliary classifiers
We present comprehensive experiments on ImageNet for addressing vanishing/exploding gradients. The papers
[35] to show the degradation problem and evaluate our of [38, 37, 31, 46] propose methods for centering layer re-
method. We show that: 1) Our extremely deep residual nets sponses, gradients, and propagated errors, implemented by
are easy to optimize, but the counterpart “plain” nets (that shortcut connections. In [43], an “inception” layer is com-
simply stack layers) exhibit higher training error when the posed of a shortcut branch and a few deeper branches.
depth increases; 2) Our deep residual nets can easily enjoy
accuracy gains from greatly increased depth, producing re- Concurrent with our work, “highway networks” [41, 42]
sults substantially better than previous networks. present shortcut connections with gating functions [15].
These gates are data-dependent and have parameters, in
Similar phenomena are also shown on the CIFAR-10 set contrast to our identity shortcuts that are parameter-free.
[20], suggesting that the optimization difficulties and the When a gated shortcut is “closed” (approaching zero), the
effects of our method are not just akin to a particular dataset. layers in highway networks represent non-residual func-
We present successfully trained models on this dataset with tions. On the contrary, our formulation always learns
over 100 layers, and explore models with over 1000 layers. residual functions; our identity shortcuts are never closed,
and all information is always passed through, with addi-
On the ImageNet classification dataset [35], we obtain tional residual functions to be learned. In addition, high-
excellent results by extremely deep residual nets. Our 152-
layer residual net is the deepest network ever presented on
ImageNet, while still having lower complexity than VGG
nets [40]. Our ensemble has 3.57% top-5 error on the
2771
way networks have not demonstrated accuracy gains with ReLU [29] and the biases are omitted for simplifying no-
extremely increased depth (e.g., over 100 layers). tations. The operation F + x is performed by a shortcut
connection and element-wise addition. We adopt the sec-
3. Deep Residual Learning ond nonlinearity after the addition (i.e., σ(y), see Fig. 2).
3.1. Residual Learning The shortcut connections in Eqn.(1) introduce neither ex-
tra parameter nor computation complexity. This is not only
Let us consider H(x) as an underlying mapping to be attractive in practice but also important in our comparisons
fit by a few stacked layers (not necessarily the entire net), between plain and residual networks. We can fairly com-
with x denoting the inputs to the first of these layers. If one pare plain/residual networks that simultaneously have the
hypothesizes that multiple nonlinear layers can asymptoti- same number of parameters, depth, width, and computa-
cally approximate complicated functions2, then it is equiv- tional cost (except for the negligible element-wise addition).
alent to hypothesize that they can asymptotically approxi-
mate the residual functions, i.e., H(x) − x (assuming that The dimensions of x and F must be equal in Eqn.(1).
the input and output are of the same dimensions). So If this is not the case (e.g., when changing the input/output
rather than expect stacked layers to approximate H(x), we channels), we can perform a linear projection Ws by the
explicitly let these layers approximate a residual function shortcut connections to match the dimensions:
F(x) := H(x) − x. The original function thus becomes
F(x)+x. Although both forms should be able to asymptot- y = F (x, {Wi}) + Wsx. (2)
ically approximate the desired functions (as hypothesized),
the ease of learning might be different. We can also use a square matrix Ws in Eqn.(1). But we will
show by experiments that the identity mapping is sufficient
This reformulation is motivated by the counterintuitive for addressing the degradation problem and is economical,
phenomena about the degradation problem (Fig. 1, left). As and thus Ws is only used when matching dimensions.
we discussed in the introduction, if the added layers can
be constructed as identity mappings, a deeper model should The form of the residual function F is flexible. Exper-
have training error no greater than its shallower counter- iments in this paper involve a function F that has two or
part. The degradation problem suggests that the solvers three layers (Fig. 5), while more layers are possible. But if
might have difficulties in approximating identity mappings F has only a single layer, Eqn.(1) is similar to a linear layer:
by multiple nonlinear layers. With the residual learning re- y = W1x + x, for which we have not observed advantages.
formulation, if identity mappings are optimal, the solvers
may simply drive the weights of the multiple nonlinear lay- We also note that although the above notations are about
ers toward zero to approach identity mappings. fully-connected layers for simplicity, they are applicable to
convolutional layers. The function F(x, {Wi}) can repre-
In real cases, it is unlikely that identity mappings are op- sent multiple convolutional layers. The element-wise addi-
timal, but our reformulation may help to precondition the tion is performed on two feature maps, channel by channel.
problem. If the optimal function is closer to an identity
mapping than to a zero mapping, it should be easier for the 3.3. Network Architectures
solver to find the perturbations with reference to an identity
mapping, than to learn the function as a new one. We show We have tested various plain/residual nets, and have ob-
by experiments (Fig. 7) that the learned residual functions in served consistent phenomena. To provide instances for dis-
general have small responses, suggesting that identity map- cussion, we describe two models for ImageNet as follows.
pings provide reasonable preconditioning.
Plain Network. Our plain baselines (Fig. 3, middle) are
3.2. Identity Mapping by Shortcuts mainly inspired by the philosophy of VGG nets [40] (Fig. 3,
left). The convolutional layers mostly have 3×3 filters and
We adopt residual learning to every few stacked layers. follow two simple design rules: (i) for the same output
A building block is shown in Fig. 2. Formally, in this paper feature map size, the layers have the same number of fil-
we consider a building block defined as: ters; and (ii) if the feature map size is halved, the num-
ber of filters is doubled so as to preserve the time com-
y = F (x, {Wi}) + x. (1) plexity per layer. We perform downsampling directly by
convolutional layers that have a stride of 2. The network
Here x and y are the input and output vectors of the lay- ends with a global average pooling layer and a 1000-way
ers considered. The function F(x, {Wi}) represents the fully-connected layer with softmax. The total number of
residual mapping to be learned. For the example in Fig. 2 weighted layers is 34 in Fig. 3 (middle).
that has two layers, F = W2σ(W1x) in which σ denotes
It is worth noticing that our model has fewer filters and
2This hypothesis, however, is still an open question. See [28]. lower complexity than VGG nets [40] (Fig. 3, left). Our 34-
layer baseline has 3.6 billion FLOPs (multiply-adds), which
is only 18% of VGG-19 (19.6 billion FLOPs).
3772
VGG-19 34-layer plain 34-layer residual Residual Network. Based on the above plain network, we
insert shortcut connections (Fig. 3, right) which turn the
image image image network into its counterpart residual version. The identity
3x3 conv, 64 shortcuts (Eqn.(1)) can be directly used when the input and
output 3x3 conv, 64 7x7 conv, 64, /2 7x7 conv, 64, /2 output are of the same dimensions (solid line shortcuts in
size: 224 pool, /2 pool, /2 Fig. 3). When the dimensions increase (dotted line shortcuts
pool, /2 in Fig. 3), we consider two options: (A) The shortcut still
output 3x3 conv, 128 3x3 conv, 64 3x3 conv, 64 performs identity mapping, with extra zero entries padded
size: 112 3x3 conv, 128 3x3 conv, 64 3x3 conv, 64 for increasing dimensions. This option introduces no extra
3x3 conv, 64 3x3 conv, 64 parameter; (B) The projection shortcut in Eqn.(2) is used to
output pool, /2 3x3 conv, 64 3x3 conv, 64 match dimensions (done by 1×1 convolutions). For both
size: 56 3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 options, when the shortcuts go across feature maps of two
3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 sizes, they are performed with a stride of 2.
3x3 conv, 256 3x3 conv, 128, /2 3x3 conv, 128, /2
3x3 conv, 256 3x3 conv, 128 3x3 conv, 128 3.4. Implementation
3x3 conv, 128 3x3 conv, 128
output pool, /2 3x3 conv, 128 3x3 conv, 128 Our implementation for ImageNet follows the practice
size: 28 3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 in [21, 40]. The image is resized with its shorter side ran-
3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 domly sampled in [256, 480] for scale augmentation [40].
3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 A 224×224 crop is randomly sampled from an image or its
3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 horizontal flip, with the per-pixel mean subtracted [21]. The
3x3 conv, 256, /2 3x3 conv, 256, /2 standard color augmentation in [21] is used. We adopt batch
output pool, /2 3x3 conv, 256 3x3 conv, 256 normalization (BN) [16] right after each convolution and
size: 14 3x3 conv, 512 3x3 conv, 256 3x3 conv, 256 before activation, following [16]. We initialize the weights
3x3 conv, 512 3x3 conv, 256 3x3 conv, 256 as in [12] and train all plain/residual nets from scratch. We
3x3 conv, 512 3x3 conv, 256 3x3 conv, 256 use SGD with a mini-batch size of 256. The learning rate
3x3 conv, 512 3x3 conv, 256 3x3 conv, 256 starts from 0.1 and is divided by 10 when the error plateaus,
3x3 conv, 256 3x3 conv, 256 and the models are trained for up to 60 × 104 iterations. We
output pool, /2 3x3 conv, 256 3x3 conv, 256 use a weight decay of 0.0001 and a momentum of 0.9. We
size: 7 3x3 conv, 256 3x3 conv, 256 do not use dropout [13], following the practice in [16].
3x3 conv, 256 3x3 conv, 256
output fc 4096 3x3 conv, 256 3x3 conv, 256 In testing, for comparison studies we adopt the standard
size: 1 fc 4096 3x3 conv, 256 3x3 conv, 256 10-crop testing [21]. For best results, we adopt the fully-
fc 1000 3x3 conv, 512, /2 3x3 conv, 512, /2 convolutional form as in [40, 12], and average the scores
3x3 conv, 512 3x3 conv, 512 at multiple scales (images are resized such that the shorter
3x3 conv, 512 3x3 conv, 512 side is in {224, 256, 384, 480, 640}).
3x3 conv, 512 3x3 conv, 512
3x3 conv, 512 3x3 conv, 512 4. Experiments
3x3 conv, 512 3x3 conv, 512
4.1. ImageNet Classification
avg pool avg pool
fc 1000 fc 1000 We evaluate our method on the ImageNet 2012 classifi-
cation dataset [35] that consists of 1000 classes. The models
Figure 3. Example network architectures for ImageNet. Left: the are trained on the 1.28 million training images, and evalu-
VGG-19 model [40] (19.6 billion FLOPs) as a reference. Mid- ated on the 50k validation images. We also obtain a final
dle: a plain network with 34 parameter layers (3.6 billion FLOPs). result on the 100k test images, reported by the test server.
Right: a residual network with 34 parameter layers (3.6 billion We evaluate both top-1 and top-5 error rates.
FLOPs). The dotted shortcuts increase dimensions. Table 1 shows
more details and other variants. Plain Networks. We first evaluate 18-layer and 34-layer
plain nets. The 34-layer plain net is in Fig. 3 (middle). The
18-layer plain net is of a similar form. See Table 1 for de-
tailed architectures.
The results in Table 2 show that the deeper 34-layer plain
net has higher validation error than the shallower 18-layer
plain net. To reveal the reasons, in Fig. 4 (left) we com-
pare their training/validation errors during the training pro-
cedure. We have observed the degradation problem - the
4773
layer name output size 18-layer 34-layer 50-layer 101-layer 152-layer
conv1 112×112
7×7, 64, stride 2
3×3 max pool, stride 2
conv2 x 56×56 3×3, 64 3×3, 64 ⎡ 1×1, 64 ⎤ ⎡ 1×1, 64 ⎤ ⎡ 1×1, 64 ⎤
3×3, 64 3×3, 64 ⎣ 3×3, 64 ×3
conv3 x 28×28 ×2 ×3 ⎣ 3×3, 64 ×3 ⎣ 3×3, 64 ×3
1×1, 256
conv4 x 14×14 1×1, 256 1×1, 256
conv5 x 7×7 3×3, 128 ×2 3×3, 128 ×4 ⎡ 1×1, 128 ⎤ ⎡ 1×1, 128 ⎤ ⎡ 1×1, 128 ⎤
1×1 3×3, 128 3×3, 128 ⎣ 3×3, 128 ×4 ⎣ 3×3, 128 ×4 ⎣ 3×3, 128 ×8
FLOPs 1×1, 512 1×1, 512 1×1, 512
3×3, 256 ×2 3×3, 256 ×6 ⎡ 1×1, 256 ⎤ ⎡ 1×1, 256 ⎤ ⎡ 1×1, 256 ⎤
3×3, 256 3×3, 256 ⎣ 3×3, 256 ×6 ⎣ 3×3, 256 ×23 ⎣ 3×3, 256 ×36
1×1, 1024 1×1, 1024 1×1, 1024
3×3, 512 ×2 3×3, 512 ⎡ 1×1, 512 ⎤ ⎡ 1×1, 512 ⎤ ⎡ 1×1, 512 ⎤
3×3, 512 3×3, 512 ×3 ⎣ 3×3, 512 ×3 ⎣ 3×3, 512 ×3 ⎣ 3×3, 512 ×3
1×1, 2048 1×1, 2048 1×1, 2048
average pool, 1000-d fc, softmax
1.8×109 3.6×109 3.8×109 7.6×109 11.3×109
Table 1. Architectures for ImageNet. Building blocks are shown in brackets (see also Fig. 5), with the numbers of blocks stacked. Down-
sampling is performed by conv3 1, conv4 1, and conv5 1 with a stride of 2.
60 60
50 50
40error (%) 40
error (%)
34-layer
18-layer
30 18-layer 30
plain-18 ResNet-18
plain-34 ResNet-34 34-layer
200 10 20 30 40 50 200 10 20 30 40 50
iter. (1e4) iter. (1e4)
Figure 4. Training on ImageNet. Thin curves denote training error, and bold curves denote validation error of the center crops. Left: plain
networks of 18 and 34 layers. Right: ResNets of 18 and 34 layers. In this plot, the residual networks have no extra parameter compared to
their plain counterparts.
18 layers plain ResNet reducing of the training error3. The reason for such opti-
34 layers 27.94 27.88 mization difficulties will be studied in the future.
28.54 25.03
Residual Networks. Next we evaluate 18-layer and 34-
Table 2. Top-1 error (%, 10-crop testing) on ImageNet validation. layer residual nets (ResNets). The baseline architectures
Here the ResNets have no extra parameter compared to their plain are the same as the above plain nets, expect that a shortcut
counterparts. Fig. 4 shows the training procedures. connection is added to each pair of 3×3 filters as in Fig. 3
(right). In the first comparison (Table 2 and Fig. 4 right),
34-layer plain net has higher training error throughout the we use identity mapping for all shortcuts and zero-padding
whole training procedure, even though the solution space for increasing dimensions (option A). So they have no extra
of the 18-layer plain network is a subspace of that of the parameter compared to the plain counterparts.
34-layer one.
We have three major observations from Table 2 and
We argue that this optimization difficulty is unlikely to Fig. 4. First, the situation is reversed with residual learn-
be caused by vanishing gradients. These plain networks are ing – the 34-layer ResNet is better than the 18-layer ResNet
trained with BN [16], which ensures forward propagated (by 2.8%). More importantly, the 34-layer ResNet exhibits
signals to have non-zero variances. We also verify that the considerably lower training error and is generalizable to the
backward propagated gradients exhibit healthy norms with validation data. This indicates that the degradation problem
BN. So neither forward nor backward signals vanish. In is well addressed in this setting and we manage to obtain
fact, the 34-layer plain net is still able to achieve compet- accuracy gains from increased depth.
itive accuracy (Table 3), suggesting that the solver works
to some extent. We conjecture that the deep plain nets may Second, compared to its plain counterpart, the 34-layer
have exponentially low convergence rates, which impact the
3We have experimented with more training iterations (3×) and still ob-
served the degradation problem, suggesting that this problem cannot be
feasibly addressed by simply using more iterations.
5774
model top-1 err. top-5 err. 64-d 256-d
VGG-16 [40] 28.07 9.33 3x3, 64
GoogLeNet [43] - 9.15 1x1, 64
PReLU-net [12] 24.27 7.38 relu relu
3x3, 64
plain-34 28.54 10.02 3x3, 64
ResNet-34 A 25.03 7.76 relu relu
ResNet-34 B 24.52 7.46
ResNet-34 C 24.19 7.40 1x1, 256
ResNet-50 22.85 6.71
ResNet-101 21.75 6.05 relu
ResNet-152 21.43 5.71
Figure 5. A deeper residual function F for ImageNet. Left: a
building block (on 56×56 feature maps) as in Fig. 3 for ResNet-
34. Right: a “bottleneck” building block for ResNet-50/101/152.
Table 3. Error rates (%, 10-crop testing) on ImageNet validation. parameter-free, identity shortcuts help with training. Next
VGG-16 is based on our test. ResNet-50/101/152 are of option B we investigate projection shortcuts (Eqn.(2)). In Table 3 we
that only uses projections for increasing dimensions. compare three options: (A) zero-padding shortcuts are used
for increasing dimensions, and all shortcuts are parameter-
method top-1 err. top-5 err. free (the same as Table 2 and Fig. 4 right); (B) projec-
VGG [40] (ILSVRC’14) - 8.43† tion shortcuts are used for increasing dimensions, and other
GoogLeNet [43] (ILSVRC’14) - 7.89 shortcuts are identity; and (C) all shortcuts are projections.
VGG [40] (v5) 7.1
PReLU-net [12] 24.4 5.71 Table 3 shows that all three options are considerably bet-
BN-inception [16] 21.59 5.81 ter than the plain counterpart. B is slightly better than A. We
ResNet-34 B 21.99 5.71 argue that this is because the zero-padded dimensions in A
ResNet-34 C 21.84 5.60 indeed have no residual learning. C is marginally better than
ResNet-50 21.53 5.25 B, and we attribute this to the extra parameters introduced
ResNet-101 20.74 4.60 by many (thirteen) projection shortcuts. But the small dif-
ResNet-152 19.87 4.49 ferences among A/B/C indicate that projection shortcuts are
19.38 not essential for addressing the degradation problem. So we
do not use option C in the rest of this paper, to reduce mem-
Table 4. Error rates (%) of single-model results on the ImageNet ory/time complexity and model sizes. Identity shortcuts are
validation set (except † reported on the test set). particularly important for not increasing the complexity of
the bottleneck architectures that are introduced below.
method top-5 err. (test)
VGG [40] (ILSVRC’14) 7.32 Deeper Bottleneck Architectures. Next we describe our
GoogLeNet [43] (ILSVRC’14) 6.66 deeper nets for ImageNet. Because of concerns on the train-
VGG [40] (v5) 6.8 ing time that we can afford, we modify the building block
PReLU-net [12] 4.94 as a bottleneck design4. For each residual function F, we
BN-inception [16] 4.82 use a stack of 3 layers instead of 2 (Fig. 5). The three layers
ResNet (ILSVRC’15) 3.57 are 1×1, 3×3, and 1×1 convolutions, where the 1×1 layers
are responsible for reducing and then increasing (restoring)
Table 5. Error rates (%) of ensembles. The top-5 error is on the dimensions, leaving the 3×3 layer a bottleneck with smaller
test set of ImageNet and reported by the test server. input/output dimensions. Fig. 5 shows an example, where
both designs have similar time complexity.
ResNet reduces the top-1 error by 3.5% (Table 2), resulting
from the successfully reduced training error (Fig. 4 right vs. The parameter-free identity shortcuts are particularly im-
left). This comparison verifies the effectiveness of residual portant for the bottleneck architectures. If the identity short-
learning on extremely deep systems. cut in Fig. 5 (right) is replaced with projection, one can
show that the time complexity and model size are doubled,
Last, we also note that the 18-layer plain/residual nets as the shortcut is connected to the two high-dimensional
are comparably accurate (Table 2), but the 18-layer ResNet ends. So identity shortcuts lead to more efficient models
converges faster (Fig. 4 right vs. left). When the net is “not for the bottleneck designs.
overly deep” (18 layers here), the current SGD solver is still
able to find good solutions to the plain net. In this case, the 50-layer ResNet: We replace each 2-layer block in the
ResNet eases the optimization by providing faster conver-
gence at the early stage. 4Deeper non-bottleneck ResNets (e.g., Fig. 5 left) also gain accuracy
from increased depth (as shown on CIFAR-10), but are not as economical
Identity vs. Projection Shortcuts. We have shown that as the bottleneck ResNets. So the usage of bottleneck designs is mainly due
to practical considerations. We further note that the degradation problem
of plain nets is also witnessed for the bottleneck designs.
6775
34-layer net with this 3-layer bottleneck block, resulting in method error (%)
a 50-layer ResNet (Table 1). We use option B for increasing 9.38
dimensions. This model has 3.8 billion FLOPs. Maxout [9] 8.81
8.22
101-layer and 152-layer ResNets: We construct 101- NIN [25]
layer and 152-layer ResNets by using more 3-layer blocks 8.39
(Table 1). Remarkably, although the depth is significantly DSN [24] 7.54 (7.72±0.16)
increased, the 152-layer ResNet (11.3 billion FLOPs) still 8.80
has lower complexity than VGG-16/19 nets (15.3/19.6 bil- # layers # params 8.75
lion FLOPs). 2.5M 7.51
FitNet [34] 19 2.3M 7.17
The 50/101/152-layer ResNets are more accurate than 1.25M 6.97
the 34-layer ones by considerable margins (Table 3 and 4). Highway [41, 42] 19 0.27M 6.43 (6.61±0.16)
We do not observe the degradation problem and thus en- 0.46M 7.93
joy significant accuracy gains from considerably increased Highway [41, 42] 32 0.66M
depth. The benefits of depth are witnessed for all evaluation 0.85M
metrics (Table 3 and 4). ResNet 20 1.7M
19.4M
Comparisons with State-of-the-art Methods. In Table 4 ResNet 32
we compare with the previous best single-model results.
Our baseline 34-layer ResNets have achieved very compet- ResNet 44
itive accuracy. Our 152-layer ResNet has a single-model
top-5 validation error of 4.49%. This single-model result ResNet 56
outperforms all previous ensemble results (Table 5). We
combine six models of different depth to form an ensemble ResNet 110
(only with two 152-layer ones at the time of submitting).
This leads to 3.57% top-5 error on the test set (Table 5). ResNet 1202
This entry won the 1st place in ILSVRC 2015.
Table 6. Classification error on the CIFAR-10 test set. All meth-
ods are with data augmentation. For ResNet-110, we run it 5 times
and show “best (mean±std)” as in [42].
4.2. CIFAR-10 and Analysis so our residual models have exactly the same depth, width,
and number of parameters as the plain counterparts.
We conducted more studies on the CIFAR-10 dataset
[20], which consists of 50k training images and 10k test- We use a weight decay of 0.0001 and momentum of 0.9,
ing images in 10 classes. We present experiments trained and adopt the weight initialization in [12] and BN [16] but
on the training set and evaluated on the test set. Our focus with no dropout. These models are trained with a mini-
is on the behaviors of extremely deep networks, but not on batch size of 128 on two GPUs. We start with a learning
pushing the state-of-the-art results, so we intentionally use rate of 0.1, divide it by 10 at 32k and 48k iterations, and
simple architectures as follows. terminate training at 64k iterations, which is determined on
a 45k/5k train/val split. We follow the simple data augmen-
The plain/residual architectures follow the form in Fig. 3 tation in [24] for training: 4 pixels are padded on each side,
(middle/right). The network inputs are 32×32 images, with and a 32×32 crop is randomly sampled from the padded
the per-pixel mean subtracted. The first layer is 3×3 convo- image or its horizontal flip. For testing, we only evaluate
lutions. Then we use a stack of 6n layers with 3×3 convo- the single view of the original 32×32 image.
lutions on the feature maps of sizes {32, 16, 8} respectively,
with 2n layers for each feature map size. The numbers of We compare n = {3, 5, 7, 9}, leading to 20, 32, 44, and
filters are {16, 32, 64} respectively. The subsampling is per- 56-layer networks. Fig. 6 (left) shows the behaviors of the
formed by convolutions with a stride of 2. The network ends plain nets. The deep plain nets suffer from increased depth,
with a global average pooling, a 10-way fully-connected and exhibit higher training error when going deeper. This
layer, and softmax. There are totally 6n+2 stacked weighted phenomenon is similar to that on ImageNet (Fig. 4, left) and
layers. The following table summarizes the architecture: on MNIST (see [41]), suggesting that such an optimization
difficulty is a fundamental problem.
output map size 32×32 16×16 8×8
Fig. 6 (middle) shows the behaviors of ResNets. Also
# layers 1+2n 2n 2n similar to the ImageNet cases (Fig. 4, right), our ResNets
manage to overcome the optimization difficulty and demon-
# filters 16 32 64 strate accuracy gains when the depth increases.
When shortcut connections are used, they are connected We further explore n = 18 that leads to a 110-layer
to the pairs of 3×3 layers (totally 3n shortcuts). On this ResNet. In this case, we find that the initial learning rate
dataset we use identity shortcuts in all cases (i.e., option A), of 0.1 is slightly too large to start converging5. So we use
0.01 to warm up the training until the training error is below
80% (about 400 iterations), and then go back to 0.1 and con-
tinue training. The rest of the learning schedule is as done
previously. This 110-layer network converges well (Fig. 6,
middle). It has fewer parameters than other deep and thin
5With an initial learning rate of 0.1, it starts converging (<90% error)
after several epochs, but still reaches similar accuracy.
7776
20 20 20
ResNet-20 residual-110
ResNet-32 residual-1202
ResNet-44
ResNet-56
56-layer ResNet-110
error (%) 10 10error (%) 10
20-layer error (%)
20-layer
5 5 110-layer 5
plain-20
plain-32
plain-44
plain-56 1
00 1 2 3 4 5 6 00 1 2 3 4 5 6 0 4 5 6
iter. (1e4) iter. (1e4) iter. (1e4)
Figure 6. Training on CIFAR-10. Dashed lines denote training error, and bold lines denote testing error. Left: plain networks. The error
of plain-110 is higher than 60% and not displayed. Middle: ResNets. Right: ResNets with 110 and 1202 layers.
3 plain-20 training data 07+12 07++12
test data VOC 07 test VOC 12 test
plain-56 VGG-16
73.2 70.4
ResNet-20 ResNet-101 76.4 73.8
std 2 ResNet-56
ResNet-110
1
0 20 40 60 80 100
layer index (original) Table 7. Object detection mAP (%) on the PASCAL VOC
2007/2012 test sets using baseline Faster R-CNN. See also ap-
3 plain-20 pendix for better results.
plain-56
ResNet-20
std 2 ResNet-56
ResNet-110
1 metric mAP@.5 mAP@[.5, .95]
VGG-16 41.5 21.2
0 20 40 60 80 100 ResNet-101 48.4 27.2
layer index (sorted by magnitude)
Figure 7. Standard deviations (std) of layer responses on CIFAR-
10. The responses are the outputs of each 3×3 layer, after BN and Table 8. Object detection mAP (%) on the COCO validation set
using baseline Faster R-CNN. See also appendix for better results.
before nonlinearity. Top: the layers are shown in their original
order. Bottom: the responses are ranked in descending order.
networks such as FitNet [34] and Highway [41] (Table 6), have similar training error. We argue that this is because of
yet is among the state-of-the-art results (6.43%, Table 6). overfitting. The 1202-layer network may be unnecessarily
large (19.4M) for this small dataset. Strong regularization
Analysis of Layer Responses. Fig. 7 shows the standard such as maxout [9] or dropout [13] is applied to obtain the
deviations (std) of the layer responses. The responses are best results ([9, 25, 24, 34]) on this dataset. In this paper, we
the outputs of each 3×3 layer, after BN and before other use no maxout/dropout and just simply impose regulariza-
nonlinearity (ReLU/addition). For ResNets, this analy- tion via deep and thin architectures by design, without dis-
sis reveals the response strength of the residual functions. tracting from the focus on the difficulties of optimization.
Fig. 7 shows that ResNets have generally smaller responses But combining with stronger regularization may improve
than their plain counterparts. These results support our ba- results, which we will study in the future.
sic motivation (Sec.3.1) that the residual functions might
be generally closer to zero than the non-residual functions. 4.3. Object Detection on PASCAL and MS COCO
We also notice that the deeper ResNet has smaller magni-
tudes of responses, as evidenced by the comparisons among Our method has good generalization performance on
ResNet-20, 56, and 110 in Fig. 7. When there are more other recognition tasks. Table 7 and 8 show the object de-
layers, an individual layer of ResNets tends to modify the tection baseline results on PASCAL VOC 2007 and 2012
signal less. [5] and COCO [26]. We adopt Faster R-CNN [32] as the de-
tection method. Here we are interested in the improvements
Exploring Over 1000 layers. We explore an aggressively of replacing VGG-16 [40] with ResNet-101. The detection
deep model of over 1000 layers. We set n = 200 that implementation (see appendix) of using both models is the
leads to a 1202-layer network, which is trained as described same, so the gains can only be attributed to better networks.
above. Our method shows no optimization difficulty, and Most remarkably, on the challenging COCO dataset we ob-
this 103-layer network is able to achieve training error tain a 6.0% increase in COCO’s standard metric (mAP@[.5,
<0.1% (Fig. 6, right). Its test error is still fairly good .95]), which is a 28% relative improvement. This gain is
(7.93%, Table 6). solely due to the learned representations.
But there are still open problems on such aggressively Based on deep residual nets, we won the 1st places in
deep models. The testing result of this 1202-layer network several tracks in ILSVRC & COCO 2015 competitions: Im-
is worse than that of our 110-layer network, although both ageNet detection, ImageNet localization, COCO detection,
and COCO segmentation. The details are in the appendix.
8777
References [28] G. Montu´far, R. Pascanu, K. Cho, and Y. Bengio. On the number of
linear regions of deep neural networks. In NIPS, 2014.
[1] Y. Bengio, P. Simard, and P. Frasconi. Learning long-term dependen-
cies with gradient descent is difficult. IEEE Transactions on Neural [29] V. Nair and G. E. Hinton. Rectified linear units improve restricted
Networks, 5(2):157–166, 1994. boltzmann machines. In ICML, 2010.
[2] C. M. Bishop. Neural networks for pattern recognition. Oxford [30] F. Perronnin and C. Dance. Fisher kernels on visual vocabularies for
university press, 1995. image categorization. In CVPR, 2007.
[3] W. L. Briggs, S. F. McCormick, et al. A Multigrid Tutorial. Siam, [31] T. Raiko, H. Valpola, and Y. LeCun. Deep learning made easier by
2000. linear transformations in perceptrons. In AISTATS, 2012.
[4] K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman. The devil [32] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards
is in the details: an evaluation of recent feature encoding methods. real-time object detection with region proposal networks. In NIPS,
In BMVC, 2011. 2015.
[5] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zis- [33] B. D. Ripley. Pattern recognition and neural networks. Cambridge
serman. The Pascal Visual Object Classes (VOC) Challenge. IJCV, university press, 1996.
pages 303–338, 2010.
[34] A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and
[6] R. Girshick. Fast R-CNN. In ICCV, 2015. Y. Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015.
[7] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hier-
[35] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,
archies for accurate object detection and semantic segmentation. In Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet
CVPR, 2014. large scale visual recognition challenge. arXiv:1409.0575, 2014.
[8] X. Glorot and Y. Bengio. Understanding the difficulty of training
deep feedforward neural networks. In AISTATS, 2010. [36] A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to
[9] I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and the nonlinear dynamics of learning in deep linear neural networks.
Y. Bengio. Maxout networks. arXiv:1302.4389, 2013. arXiv:1312.6120, 2013.
[10] K. He and J. Sun. Convolutional neural networks at constrained time
cost. In CVPR, 2015. [37] N. N. Schraudolph. Accelerated gradient descent by factor-centering
[11] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep decomposition. Technical report, 1998.
convolutional networks for visual recognition. In ECCV, 2014.
[12] K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: [38] N. N. Schraudolph. Centering neural network gradient factors. In
Surpassing human-level performance on imagenet classification. In Neural Networks: Tricks of the Trade, pages 207–226. Springer,
ICCV, 2015. 1998.
[13] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and
R. R. Salakhutdinov. Improving neural networks by preventing co- [39] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. Le-
adaptation of feature detectors. arXiv:1207.0580, 2012. Cun. Overfeat: Integrated recognition, localization and detection
[14] S. Hochreiter. Untersuchungen zu dynamischen neuronalen netzen. using convolutional networks. In ICLR, 2014.
Diploma thesis, TU Munich, 1991.
[15] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural [40] K. Simonyan and A. Zisserman. Very deep convolutional networks
computation, 9(8):1735–1780, 1997. for large-scale image recognition. In ICLR, 2015.
[16] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep
network training by reducing internal covariate shift. In ICML, 2015. [41] R. K. Srivastava, K. Greff, and J. Schmidhuber. Highway networks.
[17] H. Jegou, M. Douze, and C. Schmid. Product quantization for nearest arXiv:1505.00387, 2015.
neighbor search. TPAMI, 33, 2011.
[18] H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, and [42] R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep
C. Schmid. Aggregating local image descriptors into compact codes. networks. 1507.06228, 2015.
TPAMI, 2012.
[19] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, [43] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Er-
S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for han, V. Vanhoucke, and A. Rabinovich. Going deeper with convolu-
fast feature embedding. arXiv:1408.5093, 2014. tions. In CVPR, 2015.
[20] A. Krizhevsky. Learning multiple layers of features from tiny im-
ages. Tech Report, 2009. [44] R. Szeliski. Fast surface interpolation using hierarchical basis func-
[21] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification tions. TPAMI, 1990.
with deep convolutional neural networks. In NIPS, 2012.
[22] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, [45] R. Szeliski. Locally adapted hierarchical basis preconditioning. In
W. Hubbard, and L. D. Jackel. Backpropagation applied to hand- SIGGRAPH, 2006.
written zip code recognition. Neural computation, 1989.
[23] Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Mu¨ller. Efficient backprop. [46] T. Vatanen, T. Raiko, H. Valpola, and Y. LeCun. Pushing stochas-
In Neural Networks: Tricks of the Trade, pages 9–50. Springer, 1998. tic gradient towards second-order methods–backpropagation learn-
[24] C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu. Deeply- ing with transformations in nonlinearities. In Neural Information
supervised nets. arXiv:1409.5185, 2014. Processing, 2013.
[25] M. Lin, Q. Chen, and S. Yan. Network in network. arXiv:1312.4400,
2013. [47] A. Vedaldi and B. Fulkerson. VLFeat: An open and portable library
[26] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, of computer vision algorithms, 2008.
P. Dolla´r, and C. L. Zitnick. Microsoft COCO: Common objects in
context. In ECCV. 2014. [48] W. Venables and B. Ripley. Modern applied statistics with s-plus.
[27] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks 1999.
for semantic segmentation. In CVPR, 2015.
[49] M. D. Zeiler and R. Fergus. Visualizing and understanding convolu-
tional neural networks. In ECCV, 2014.
9778

BIN
demo/page_imgs/page_7.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 93 KiB