同步完整源码 - 2026-05-25

This commit is contained in:
xiaoxue
2026-05-25 00:32:43 +08:00
commit 77248c0d06
46 changed files with 1806 additions and 0 deletions
+21
View File
@@ -0,0 +1,21 @@
# Evitar subir archivos .zip
/Salidas_archivos/*.zip
# Environment
.venv/
# Configuracion
config.json
# Cache
__pycache__/
# Logs
logs/
*.log
#Tests
.pytest_tmp/
.pytest_cache/
.coverage
htmlcov/
Binary file not shown.
File diff suppressed because one or more lines are too long
@@ -0,0 +1,68 @@
%PDF-1.3
%“Œ‹ž ReportLab Generated PDF document http://www.reportlab.com
1 0 obj
<<
/F1 2 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/Contents 7 0 R /MediaBox [ 0 0 612 792 ] /Parent 6 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
4 0 obj
<<
/PageMode /UseNone /Pages 6 0 R /Type /Catalog
>>
endobj
5 0 obj
<<
/Author (anonymous) /CreationDate (D:20260203033950+00'00') /Creator (ReportLab PDF Library - www.reportlab.com) /Keywords () /ModDate (D:20260203033950+00'00') /Producer (ReportLab PDF Library - www.reportlab.com)
/Subject (unspecified) /Title (untitled) /Trapped /False
>>
endobj
6 0 obj
<<
/Count 1 /Kids [ 3 0 R ] /Type /Pages
>>
endobj
7 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 126
>>
stream
GapQh0E=F,0U\H3T\pNYT^QKk?tc>IP,;W#U1^23ihPEM_M(M8&8Hm/&J,g.K]FUr"Ltrj'PHPWAEG59Vrjfk/*JGskNCnVPb_?EjQgM0KOcNbBS!N@DufqXC*%!~>endstream
endobj
xref
0 8
0000000000 65535 f
0000000073 00000 n
0000000104 00000 n
0000000211 00000 n
0000000404 00000 n
0000000472 00000 n
0000000768 00000 n
0000000827 00000 n
trailer
<<
/ID
[<2223e39864f5271a4fa9ba223bc69a84><2223e39864f5271a4fa9ba223bc69a84>]
% ReportLab generated PDF document -- digest (http://www.reportlab.com)
/Info 5 0 R
/Root 4 0 R
/Size 8
>>
startxref
1043
%%EOF
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
this is not a valid pptx zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1 @@
not a real xlsx
Binary file not shown.
Binary file not shown.
+27
View File
@@ -0,0 +1,27 @@
---
## LICENSE
```txt
MIT License
Copyright (c) 2026 ING.Jose Leonardo Murillo Avalos
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+147
View File
@@ -0,0 +1,147 @@
# Document-Image-Extractor
CLI tool to extract embedded images from **DOCX**, **PDF**, **PPTX** and **XLSX** files, with deduplication , size filtering, and batch export to ZIPs.
---
## Features:
Extract images from:
- DOCX (Word documents)
- PDF (documents)
- PPTX (Powerpoint documents)
- XLSX (Excel documents)
Outputs:
- Creates a **ZIP per input file** with extracted images
built-in helpers:
- **Deduplication** (skips repeated images within the same document)
- **Size filter** (`min_kb` default is 5kb)
- Handles “no images” and **corrupt files** gracefully
---
## Project status
this repository is begin improved **phase by phase**
---
## Requirements
- python 3.12+ (recomended)
Dependencies (install from 'requirements.txt'):
- python-docx
- PyMuPDF
- pillow
## Installation
### 1. Clone the repository
```bash
git clone https://github.com/LeoMurilloDev/document-image-extractor.git
cd document-image-extractor
```
### 2. Create and activate a virtual environment
#### Windows
```bash
python -m venv .venv
.\.venv\Scripts\activate
```
#### macOS / Linux
```bash
python3 -m venv .venv
source .venv/bin/activate
```
### 3. Install dependencies
pip install -r requirements.txt
## Usage
### Folder structure expected by the script
the script creates these folders automatically if they don't exist:
- **Entrdas_archivos/** -> place your **.docx** and **.pdf** files here
- **Salidas_archivos/** -> output ZIPs will be generated here
- **temp/** -> temporary extraction folder (auto-cleaned)
### Configuration
You can customize filters without editing the code using `config.json` (repo root).
Example:
```json
{
"filters": {
"min_kb": 5,
"min_width": 0,
"min_height": 0
}
}
```
- `min_kb`: minimum file size in kb (default: 5)
- `min_width`/ `min_height`: optional dimension filter (0 disables it)
## Run
```bash
python main.py
```
## CLI usage
The tool can be used with default folders/config:
```bash
python main.py
python main.py --input Entradas_archivos --output Salidas_archivos
python main.py --input example.pptx --output Salidas_archivos
python main.py --input Entradas_archivos --recursive
python main.py --input Entradas_archivos --min-kb 1 --min-width 100 --min-height 100
python main.py --input Entradas_archivos --no-dedup
python main.py --input Entradas_archivos --format folder
python main.py --input Entradas_archivos --log-level DEBUG --log-file logs/debug.log
```
## Output
- For each input file, a ZIP is created in **Salidas_archivos/**
- Example:
- Input: **Entradas_archivos/report.pdf**
- Output: **Salidas_archivos/report.zip**
## What to expect
When you run the script, it prints a summary per file:
- `guardadas` -> images saved successfully
- `duplicadas` -> images skipped due to hash duplication
- `pequeñas` -> images filtered out by size
- `encontradas` -> images found inside the document
### Important notes
- In `DOCX`, images are saved using the real extension (.jpg, .png, .gif, etc)
- `temp/` is cleaned even when a file fails
## Test suites
we use small test suites to validate.
### Documents to try
Includes:
- Mixed formats (JPG/PNG/GIF)
- Duplicates
- Small icon filtered out by size
- Corrupt files (error handling)
Manual validation steps:
1. Copy test files into `Entradas_archivos/`
2. Run `python main.py`
3. Verify
- Output ZIPs exist in `Salidas_archivos/`
- Extencions are correct in DOCX resutls (.jpg, .png, .gif)
- Duplicates are removed
- `temp/` is empty at the end
## Contributing
if you want to propose changes:
1. Fork the repo
2. Create a branch
3. Open a PR with a clear description
+20
View File
@@ -0,0 +1,20 @@
{
"paths": {
"input_dir": "Entradas_archivos",
"output_dir": "Salidas_archivos",
"temp_dir": "temp"
},
"filters": {
"min_kb": 5,
"min_width": 0,
"min_height": 0
},
"dedup": {"enabled": true},
"output": {"format": "zip"},
"logging": {
"level": "INFO",
"log_file": "logs/run.log",
"format": "%(asctime)s | %(levelname)s | %(message)s",
"datefmt": "%Y-%m-%d %H:%M:%S"
}
}
+2
View File
@@ -0,0 +1,2 @@
__version__ = "0.1"
__all__ = ["config", "pipeline", "report"]
+162
View File
@@ -0,0 +1,162 @@
import argparse
from typing import Any, Dict
from . import __version__
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="document-image-extractor",
description=(
"Extract embedded images from PDF, DOCX, PPTX and XLSX files. "
"Supports deduplication, filters and ZIP/folder output."
),
)
parser.add_argument(
"-c",
"--config",
dest="config_path",
default="config.json",
help="Ruta del archivo de configuración. Default: config.json",
)
parser.add_argument(
"-i",
"--input",
dest="input_path",
default=None,
help="Archivo o carpeta de entrada. Default: paths.input_dir del config.",
)
parser.add_argument(
"-o",
"--output",
dest="output_dir",
default=None,
help="Carpeta de salida. Default: paths.output_dir del config.",
)
parser.add_argument(
"--temp-dir",
dest="temp_dir",
default=None,
help="Carpeta temporal. Default: paths.temp_dir del config.",
)
parser.add_argument(
"-r",
"--recursive",
action="store_true",
help="Procesa archivos dentro de subcarpetas cuando --input es una carpeta.",
)
parser.add_argument(
"--min-kb",
dest="min_kb",
type=int,
default=None,
help="Tamaño mínimo en KB para guardar una imagen.",
)
parser.add_argument(
"--min-width",
dest="min_width",
type=int,
default=None,
help="Ancho mínimo en pixeles. 0 desactiva el filtro.",
)
parser.add_argument(
"--min-height",
dest="min_height",
type=int,
default=None,
help="Alto mínimo en pixeles. 0 desactiva el filtro.",
)
dedup_group = parser.add_mutually_exclusive_group()
dedup_group.add_argument(
"--dedup",
dest="dedup_enabled",
action="store_true",
help="Activa deduplicación por hash.",
)
dedup_group.add_argument(
"--no-dedup",
dest="dedup_enabled",
action="store_false",
help="Desactiva deduplicación por hash.",
)
parser.set_defaults(dedup_enabled=None)
parser.add_argument(
"--format",
dest="output_format",
choices=["zip", "folder"],
default=None,
help="Formato de salida. Opciones: zip, folder. Default: config output.format.",
)
parser.add_argument(
"--log-level",
dest="log_level",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
default=None,
help="Nivel de logs.",
)
parser.add_argument(
"--log-file",
dest="log_file",
default=None,
help="Ruta del archivo de logs. Ejemplo: logs/run.log",
)
parser.add_argument(
"--version",
action="version",
version=f"%(prog)s {__version__}",
)
return parser.parse_args()
def apply_cli_overrides(cfg: Dict[str, Any], args: argparse.Namespace) -> Dict[str, Any]:
cfg.setdefault("runtime", {})
# Input runtime: puede ser archivo o carpeta
cfg["runtime"]["input_path"] = args.input_path or cfg["paths"]["input_dir"]
cfg["runtime"]["input_was_provided"] = bool(args.input_path)
cfg["runtime"]["recursive"] = bool(args.recursive)
# Paths
if args.output_dir is not None:
cfg["paths"]["output_dir"] = args.output_dir
if args.temp_dir is not None:
cfg["paths"]["temp_dir"] = args.temp_dir
# Filters
if args.min_kb is not None:
cfg["filters"]["min_kb"] = int(args.min_kb)
if args.min_width is not None:
cfg["filters"]["min_width"] = int(args.min_width)
if args.min_height is not None:
cfg["filters"]["min_height"] = int(args.min_height)
# Dedup
if args.dedup_enabled is not None:
cfg["dedup"]["enabled"] = bool(args.dedup_enabled)
# Output
if args.output_format is not None:
cfg["output"]["format"] = args.output_format
# Logging
if args.log_level is not None:
cfg["logging"]["level"] = args.log_level
if args.log_file is not None:
cfg["logging"]["log_file"] = args.log_file
return cfg
+54
View File
@@ -0,0 +1,54 @@
import json
from pathlib import Path
from typing import Any, Dict
DEFAULT_CONFIG: Dict[str, Any] = {
"paths": {
"input_dir": "Entradas_archivos",
"output_dir": "Salidas_archivos",
"temp_dir": "temp",
},
"filters": {
"min_kb": 5,
"min_width": 0,
"min_height": 0,
},
"dedup": {"enabled": True},
"output": {"format": "zip"},
"logging": {
"level": "INFO",
"log_file": "logs/run.log",
"format": "%(asctime)s | %(levelname)s | %(message)s",
"datefmt": "%Y-%m-%d %H:%M:%S"
}
}
# Cargar la configuracion del programa y parametros basicos
def load_config(config_path: Path = Path("config.json")) -> Dict[str, Any]:
# Carga de archivo config.json y en caso de configuraciones faltantes he hace mescla con DEFAULT_CONFIG
cfg = json.loads(json.dumps(DEFAULT_CONFIG))
if not config_path.exists(): return cfg
try:
user_cfg = json.loads(config_path.read_text(encoding="utf-8"))
except Exception:
return cfg
for section in ("paths", "filters", "dedup", "output", "logging"):
if isinstance(user_cfg.get(section), dict): cfg[section].update(user_cfg[section])
# Normalizacion minima
cfg["filters"]["min_kb"] = int(cfg["filters"].get("min_kb", 5))
cfg["filters"]["min_width"] = int(cfg["filters"].get("min_width", 0))
cfg["filters"]["min_height"] = int(cfg["filters"].get("min_height", 0))
cfg["dedup"]["enabled"] = bool(cfg["dedup"].get("enabled", True))
cfg["output"]["format"] = str(cfg["output"].get("format", "zip")).lower()
cfg["logging"]["level"] = str(cfg["logging"].get("level", "INFO")).upper()
cfg["logging"]["format"] = str(cfg["logging"].get("format", DEFAULT_CONFIG["logging"]["format"]))
cfg["logging"]["datefmt"] = str(cfg["logging"].get("datefmt", DEFAULT_CONFIG["logging"]["datefmt"]))
log_file = str(cfg["logging"].get("log_file", "")).strip()
if not log_file:
log_file = DEFAULT_CONFIG["logging"]["log_file"]
cfg["logging"]["log_file"] = log_file
return cfg
@@ -0,0 +1 @@
__all__ = ["docx_extractor", "pdf_extractor","pptx_extractor","xlsx_extractor","ooxml_media_extractor"]
@@ -0,0 +1,70 @@
from pathlib import Path
from typing import Any, Dict, Set
from docx import Document
from ..utils.hashing import md5_bytes
from ..utils.files import is_small_kb
from ..utils.images import get_image_size, fails_dimension_filter, normalize_ext
def _docx_image_ext(image_part) -> str:
ct = getattr(image_part, "content_type", None)
if ct and "/" in ct:
return normalize_ext(ct.split("/")[-1])
partname = str(getattr(image_part, "partname", "")).lower()
if "." in partname:
return normalize_ext(partname.split(".")[-1])
return "bin"
def extract_docx_images(docx_path: Path, temp_folder: Path, cfg: Dict[str, Any]) -> Dict[str, int]:
filters = cfg["filters"]
min_kb = int(filters["min_kb"])
min_w = int(filters["min_width"])
min_h = int(filters["min_height"])
dedup_enabled = bool(cfg["dedup"]["enabled"])
stats: Dict[str, int] = {"found": 0, "saved": 0, "duplicates": 0, "filtered_small": 0, "filtered_dims": 0, "errors": 0}
doc = Document(str(docx_path))
image_parts = list(doc.part.package.image_parts)
stats["found"] = len(image_parts)
hashes: Set[str] = set()
for i, image_part in enumerate(image_parts, start=1):
out_path = None
try:
blob = image_part.blob
digest = md5_bytes(blob)
if dedup_enabled and digest in hashes:
stats["duplicates"] += 1
continue
ext = _docx_image_ext(image_part)
out_path = temp_folder / f"image_{i:03d}.{ext}"
out_path.write_bytes(blob)
if is_small_kb(out_path, min_kb):
stats["filtered_small"] += 1
out_path.unlink(missing_ok=True)
continue
dims = get_image_size(out_path)
if fails_dimension_filter(dims, min_w, min_h):
stats["filtered_dims"] += 1
out_path.unlink(missing_ok=True)
continue
if dedup_enabled:
hashes.add(digest)
stats["saved"] += 1
except Exception:
stats["errors"] += 1
if out_path is not None:
try:
out_path.unlink(missing_ok=True)
except Exception:
pass
return stats
@@ -0,0 +1,67 @@
import zipfile
from pathlib import Path
from typing import Any, Dict, Iterable, Set, Tuple
from ..utils.hashing import md5_bytes
from ..utils.files import is_small_kb
from ..utils.images import get_image_size, fails_dimension_filter, normalize_ext
IMAGE_EXTS = {"png", "jpg", "jpeg", "gif", "bmp", "tiff", "tif", "webp", "emf", "wmf", "svg"}
def extract_ooxml_media_images(archive_path: Path, media_prefixes: Tuple[str, ...], temp_folder: Path, cfg: Dict[str, Any], source_tag: str) -> Dict[str, int]:
# Extraer imagenes desde un OOXML (pptx, xlsx) leyendo lor archivos en media/
filters = cfg["filters"]
min_kb = int(filters["min_kb"])
min_w = int(filters["min_width"])
min_h = int(filters["min_height"])
dedup_enabled = bool(cfg["dedup"]["enabled"])
stats: Dict[str, int] = {"found": 0, "saved": 0, "duplicates": 0, "filtered_small": 0, "filtered_dims": 0, "errors": 0}
hashes: Set[str] = set()
counter = 0
with zipfile.ZipFile(archive_path, "r") as zf:
for info in zf.infolist():
name = info.filename
if name.endswith("/"): continue
if not any(name.startswith(prefix) for prefix in media_prefixes): continue
ext = normalize_ext(Path(name).suffix.replace(".",""))
if not ext or ext not in IMAGE_EXTS: continue
out_path = None
try:
data = zf.read(name)
stats["found"] += 1
digest = md5_bytes(data)
if dedup_enabled and digest in hashes:
stats["duplicates"] += 1
continue
counter += 1
out_path = temp_folder / f"image_{counter:03d}_{source_tag}.{ext}"
out_path.write_bytes(data)
if is_small_kb(out_path, min_kb):
stats["filtered_small"] += 1
out_path.unlink(missing_ok=True)
continue
dims = get_image_size(out_path)
if fails_dimension_filter(dims, min_w, min_h):
stats["filtered_dims"] += 1
out_path.unlink(missing_ok=True)
continue
if dedup_enabled: hashes.add(digest)
stats["saved"] += 1
except Exception:
stats["errors"] += 1
if out_path is not None:
try:
out_path.unlink(missing_ok=True)
except Exception:
pass
return stats
@@ -0,0 +1,68 @@
from pathlib import Path
from typing import Any, Dict, Set
import fitz
from ..utils.hashing import md5_bytes
from ..utils.files import is_small_kb
from ..utils.images import get_image_size, fails_dimension_filter, normalize_ext
def extract_pdf_images(pdf_path: Path, temp_folder: Path, cfg: Dict[str, Any]) -> Dict[str, int]:
# Extraer las imagenes de vienen en el documento pdf
filters = cfg["filters"]
min_kb = filters["min_kb"]
min_w = filters["min_width"]
min_h = filters["min_height"]
dedup_enabled = cfg["dedup"]["enabled"]
stats = {"found": 0,"saved": 0, "duplicates": 0, "filtered_small": 0, "filtered_dims": 0, "errors": 0}
hashes: Set[str] = set()
counter = 0
with fitz.open(str(pdf_path)) as pdf:
for page_index in range(len(pdf)):
page = pdf[page_index]
for img in page.get_images(full=True):
out_path = None
try:
xref = img[0]
base = pdf.extract_image(xref)
data = base.get("image", b"")
ext = normalize_ext(base.get("ext", "bin"))
stats["found"] += 1
digest = md5_bytes(data)
if dedup_enabled and digest in hashes:
stats["duplicates"] += 1
continue
counter += 1
out_path = temp_folder / f"image_{counter:03d}_p{page_index+1:03d}.{ext}"
out_path.write_bytes(data)
# Filtro KB
if is_small_kb(out_path, min_kb):
stats["filtered_small"] += 1
out_path.unlink(missing_ok=True)
continue
# Filtro de dimenciones
dims = get_image_size(out_path)
if fails_dimension_filter(dims, min_w, min_h):
stats["filtered_dims"] += 1
out_path.unlink(missing_ok=True)
continue
if dedup_enabled:
hashes.add(digest)
stats["saved"] += 1
except Exception:
stats["errors"] += 1
if out_path is not None:
try:
out_path.unlink(missing_ok=True)
except Exception:
pass
return stats
@@ -0,0 +1,12 @@
from pathlib import Path
from typing import Any, Dict
from .ooxml_media_extractor import extract_ooxml_media_images
def extract_pptx_images(pptx_path: Path, temp_folder: Path, cfg: Dict[str, Any]) -> Dict[str, int]:
return extract_ooxml_media_images(archive_path=pptx_path,
media_prefixes=("ppt/media/",),
temp_folder=temp_folder,
cfg=cfg,
source_tag="pptx")
@@ -0,0 +1,12 @@
from pathlib import Path
from typing import Any, Dict
from .ooxml_media_extractor import extract_ooxml_media_images
def extract_xlsx_images(xlsx_path: Path, temp_folder: Path, cfg: Dict[str, Any]) -> Dict[str, int]:
return extract_ooxml_media_images(
archive_path=xlsx_path,
media_prefixes=("xl/media/",),
temp_folder=temp_folder,
cfg=cfg,
source_tag="xlsx",
)
+20
View File
@@ -0,0 +1,20 @@
import logging
from pathlib import Path
from typing import Any, Dict
def setup_logging(cfg: Dict[str, Any]) -> None:
log_cfg = cfg.get("logging", {}) if isinstance(cfg, dict) else {}
level_name = str(log_cfg.get("level", "INFO")).upper()
level = getattr(logging, level_name, logging.INFO)
fmt = str(log_cfg.get("format", "%(asctime)s | %(levelname)s | %(message)s"))
datefmt = str(log_cfg.get("datefmt", "%Y-%m-%d %H:%M:%S"))
log_file = str(log_cfg.get("log_file", "logs/run.log")).strip() or "logs/run.log"
log_path = Path(log_file)
log_path.parent.mkdir(parents=True, exist_ok=True)
handlers = [logging.StreamHandler(), logging.FileHandler(log_path, encoding="utf-8")]
logging.basicConfig(level=level, format=fmt, datefmt=datefmt, handlers=handlers, force=True)
+74
View File
@@ -0,0 +1,74 @@
from pathlib import Path
from typing import Any, Dict, List
from .extractors.docx_extractor import extract_docx_images
from .extractors.pdf_extractor import extract_pdf_images
from .extractors.pptx_extractor import extract_pptx_images
from .extractors.xlsx_extractor import extract_xlsx_images
from .utils.files import clean_dir, ensure_dir, create_zip_from_folder, copy_folder
SUPPORTED_EXTS = {".pdf", ".docx", ".pptx", ".xlsx"}
def process_file(file_path: Path, cfg: Dict[str, Any]) -> Dict:
paths = cfg["paths"]
temp_root = Path(paths["temp_dir"])
output_dir = Path(paths["output_dir"])
output_format = cfg["output"]["format"]
ensure_dir(temp_root)
ensure_dir(output_dir)
name = file_path.stem
temp_folder = temp_root / name
clean_dir(temp_folder)
ensure_dir(temp_folder)
try:
ext = file_path.suffix.lower()
if ext == ".pdf":
stats = extract_pdf_images(file_path, temp_folder, cfg)
elif ext == ".docx":
stats = extract_docx_images(file_path, temp_folder, cfg)
elif ext == ".pptx":
stats = extract_pptx_images(file_path, temp_folder, cfg)
elif ext == ".xlsx":
stats = extract_xlsx_images(file_path, temp_folder, cfg)
else:
return {"skipped": True, "reason": f"extencion no soportada ({ext})"}
if stats["saved"] > 0:
if output_format == "zip":
zip_path = output_dir / f"{name}.zip"
create_zip_from_folder(temp_folder, zip_path)
elif output_format == "folder":
folder_path = output_dir / name
copy_folder(temp_folder, folder_path)
else:
return {"error": f"formato de salida no soportado: {output_format}"}
return stats
except Exception as e:
return {"error": str(e)}
finally:
clean_dir(temp_folder)
def list_input_files(input_path: Path, recursive: bool = False) -> List[Path]:
if input_path.is_file():
if input_path.suffix.lower() in SUPPORTED_EXTS:
return [input_path]
return []
if not input_path.is_dir():
return []
pattern = "**/*" if recursive else "*"
files = [
f for f in input_path.glob(pattern)
if f.is_file() and f.suffix.lower() in SUPPORTED_EXTS
]
return sorted(files)
+52
View File
@@ -0,0 +1,52 @@
import logging
from typing import Dict
logger = logging.getLogger("document_image_extractor")
def log_file_report(filename: str, stats: Dict) -> None:
if stats.get("skipped"):
logger.warning("%s: %s", filename, stats.get("reason"))
return
if "error" in stats:
logger.error("%s: %s", filename, stats["error"])
return
if stats.get("saved", 0) > 0:
logger.info(
"%s | found=%s, saved=%s, dupes=%s, filtered_small=%s, filtered_dims=%s, errors=%s",
filename,
stats.get("found", 0),
stats.get("saved", 0),
stats.get("duplicates", 0),
stats.get("filtered_small", 0),
stats.get("filtered_dims", 0),
stats.get("errors", 0),
)
else:
logger.warning(
"%s | No hay imágenes válidas | found=%s, dupes=%s, filtered_small=%s, filtered_dims=%s, errors=%s",
filename,
stats.get("found", 0),
stats.get("duplicates", 0),
stats.get("filtered_small", 0),
stats.get("filtered_dims", 0),
stats.get("errors", 0),
)
def accumulate_totals(total: Dict[str, int], stats: Dict) -> None:
for k in total:
total[k] += int(stats.get(k, 0))
def log_summary(files: int, skipped: int, failed: int, total: Dict[str, int]) -> None:
logger.info("=== RESUMEN ===")
logger.info(
"files=%s, skipped=%s, failed=%s | found=%s, saved=%s, dupes=%s, filtered_small=%s, filtered_dims=%s, errors=%s",
files, skipped, failed,
total.get("found", 0),
total.get("saved", 0),
total.get("duplicates", 0),
total.get("filtered_small", 0),
total.get("filtered_dims", 0),
total.get("errors", 0),
)
@@ -0,0 +1 @@
__all_ = ["files", "hashing", "images"]
+27
View File
@@ -0,0 +1,27 @@
import shutil
import zipfile
from pathlib import Path
def ensure_dir(path: Path) -> None:
path.mkdir(parents=True, exist_ok=True)
def clean_dir(path: Path) -> None:
# Eliminar archivos de la carpeta esfecifica
if path.exists(): shutil.rmtree(path, ignore_errors=True)
def create_zip_from_folder(folder: Path, zip_destine: Path) -> None:
# Crear un zip con el contenido de folder
with zipfile.ZipFile(zip_destine, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
for file in folder.rglob("*"):
if file.is_file(): zf.write(file, arcname=file.relative_to(folder))
def is_small_kb(path: Path, min_kb: int) -> bool:
# Validacion booleana si la imagen es mas chica que el parametro min_kb configurado
return (path.stat().st_size / 1024) < min_kb
def copy_folder(src: Path, dest: Path) -> None:
# Copia una carpeta al destino, si el destino existe lo remplaza
if dest.exists(): shutil.rmtree(dest, ignore_errors=True)
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copytree(src, dest)
@@ -0,0 +1,4 @@
import hashlib
def md5_bytes(data: bytes) -> str:
return hashlib.md5(data).hexdigest()
+32
View File
@@ -0,0 +1,32 @@
from pathlib import Path
from typing import Optional, Tuple
from PIL import Image
def get_image_size(path: Path) -> Optional[Tuple[int, int]]:
# Devolver el ancho y el alto de las imagenes, En caso de no poder abrir regresa None
try:
with Image.open(path) as im:
return im.size
except Exception:
return None
def fails_dimension_filter(dims: Optional[Tuple[int, int]], min_w: int, min_h: int) -> bool:
# En caso de que dims sea None se omiten validaciones
if min_w <= 0 and min_h <= 0: return False
if dims is None: return False
w, h = dims
if min_w > 0 and w < min_w: return True
if min_h > 0 and h < min_h: return True
return False
def normalize_ext(ext: str) -> str:
# Normalizacion de extenciones para imagenes
ext = (ext or "bin").lower().strip(".")
if ext in ("jpeg", "jpe"):
return "jpg"
if ext == "tif":
return "tiff"
return ext
+80
View File
@@ -0,0 +1,80 @@
import logging
from pathlib import Path
from document_image_extractor.cli import parse_args, apply_cli_overrides
from document_image_extractor.config import load_config
from document_image_extractor.logging_setup import setup_logging
from document_image_extractor.pipeline import list_input_files, process_file, SUPPORTED_EXTS
from document_image_extractor.report import log_file_report, accumulate_totals, log_summary
from document_image_extractor.utils.files import ensure_dir
logger = logging.getLogger("document_image_extractor")
def main():
args = parse_args()
cfg = load_config(Path(args.config_path))
cfg = apply_cli_overrides(cfg, args)
setup_logging(cfg)
paths = cfg["paths"]
runtime = cfg["runtime"]
input_path = Path(runtime["input_path"])
output_dir = Path(paths["output_dir"])
temp_dir = Path(paths["temp_dir"])
recursive = bool(runtime["recursive"])
if not input_path.exists():
if runtime.get("input_was_provided"):
logger.error("La ruta de entrada no existe: %s", input_path)
return
ensure_dir(input_path)
ensure_dir(output_dir)
ensure_dir(temp_dir)
logger.debug("Input path: %s", input_path.resolve())
logger.debug("Output dir: %s", output_dir.resolve())
logger.debug("Temp dir: %s", temp_dir.resolve())
logger.debug("Recursive: %s", recursive)
logger.debug("Supported extensions: %s", ", ".join(sorted(SUPPORTED_EXTS)))
logger.debug(
"Filters: min_kb=%s, min_width=%s, min_height=%s",
cfg["filters"]["min_kb"],
cfg["filters"]["min_width"],
cfg["filters"]["min_height"],
)
logger.debug("Output format: %s", cfg["output"]["format"])
logger.debug("Dedup enabled: %s", cfg["dedup"]["enabled"])
files = list_input_files(input_path, recursive=recursive)
if not files:
logger.warning(
"No se encontraron archivos soportados (%s) en: %s",
", ".join(sorted(SUPPORTED_EXTS)),
input_path.resolve(),
)
return
total = {"found": 0, "saved": 0, "duplicates": 0, "filtered_small": 0, "filtered_dims": 0, "errors": 0}
skipped = 0
failed = 0
for f in files:
stats = process_file(f, cfg)
log_file_report(f.name, stats)
if stats.get("skipped"):
skipped += 1
continue
if "error" in stats:
failed += 1
continue
accumulate_totals(total, stats)
log_summary(files=len(files), skipped=skipped, failed=failed, total=total)
if __name__== "__main__":
main()
+4
View File
@@ -0,0 +1,4 @@
[pytest]
testpaths = tests
python_files = test_*.py
addopts = -ra --basetemp=.pytest_tmp
+1
View File
@@ -0,0 +1 @@
pytest==8.0
+3
View File
@@ -0,0 +1,3 @@
python-docx==1.2.0
PyMuPDF==1.26.7
pillow==12.1.0
View File
+132
View File
@@ -0,0 +1,132 @@
import zipfile
from pathlib import Path
from typing import Any, Dict
import fitz
import pytest
from docx import Document
from PIL import Image, ImageDraw
@pytest.fixture
def sample_images(tmp_path: Path) -> Dict[str, Path]:
assets = tmp_path / "assets"
assets.mkdir()
big_png = assets / "big_simple.png"
small_icon = assets / "small_icon.png"
photo_jpg = assets / "photo.jpg"
# Imagen grande y comprimible
img = Image.new("RGB", (900, 600), (245, 245, 245))
draw = ImageDraw.Draw(img)
draw.rectangle([40, 40, 860, 560], outline=(0, 0, 0), width=6)
draw.text((60, 60), "BIG PNG", fill=(0, 0, 0))
img.save(big_png, format="PNG", optimize=True)
# Icono Pequeño
icon = Image.new("RGB", (32, 32), (255, 255, 255))
draw = ImageDraw.Draw(icon)
draw.rectangle([2, 2, 29, 29], outline=(0, 0, 0), width=2)
draw.line([6, 16, 26, 16], fill=(0, 0, 0), width=2)
draw.line([16, 6, 16, 26], fill=(0, 0, 0), width=2)
icon.save(small_icon, format="PNG", compress_level=0)
# Imagen JPG Mediana
jpg = Image.new("RGB", (700, 450), (255, 255, 255))
draw = ImageDraw.Draw(jpg)
for i in range(20):
x0 = 20 + i * 12
y0 = 40 + i * 8
x1 = x0 + 160
y1 = y0 + 80
draw.rectangle([x0, y0, x1, y1], outline=(30, 90, 180), width=3)
draw.text((40, 40), "PHOTO JPG", fill=(0, 0, 0))
jpg.save(photo_jpg, format="JPEG", quality=85, optimize=True)
return {
"big_png": big_png,
"small_icon": small_icon,
"photo_jpg": photo_jpg,
}
@pytest.fixture
def base_cfg(tmp_path: Path) -> Dict[str, Dict]:
return {
"paths": {
"input_dir": str(tmp_path / "input"),
"output_dir": str(tmp_path / "output"),
"temp_dir": str(tmp_path / "temp"),
},
"filters": {
"min_kb": 1,
"min_width": 0,
"min_height": 0,
},
"dedup": {
"enabled": True,
},
"output": {
"format": "zip",
},
"logging": {
"level": "INFO",
"log_file": str(tmp_path / "logs" / "run.log"),
"format": "%(asctime)s | %(levelname)s | %(message)s",
"datefmt": "%Y-%m-%d %H:%M:%S",
},
}
# Documento de prueba DOCX
def create_docx(path: Path, image_paths: list[Path]) -> Path:
doc = Document()
doc.add_heading("TEST DOCX", level=1)
for image_path in image_paths:
doc.add_paragraph(image_path.name)
doc.add_picture(str(image_path))
doc.save(str(path))
return path
# Documento de prueba PDF
def create_pdf(path: Path, image_path: Path) -> Path:
doc = fitz.open()
page = doc.new_page(width=595, height=842)
rect = fitz.Rect(72, 72, 420, 360)
page.insert_image(rect, stream=image_path.read_bytes())
doc.save(str(path))
doc.close()
return path
# Documento de prueba PPTX
def create_fake_pptx(path: Path, media: dict[str, bytes]) -> Path:
with zipfile.ZipFile(path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr("[Content_Types].xml", "<Types></Types>")
for filename, data in media.items():
zf.writestr(f"ppt/media/{filename}", data)
return path
# Documento de prueba XLSX
def create_fake_xlsx(path: Path, media: dict[str, bytes]) -> Path:
with zipfile.ZipFile(path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr("[Content_Types].xml", "<Types></Types>")
for filename, data in media.items():
zf.writestr(f"xl/media/{filename}", data)
return path
# Creacion XML Base
def create_empty_ooxml(path: Path) -> Path:
with zipfile.ZipFile(path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr("[Content_Types].xml", "<Types></Types>")
return path
def zip_names(zip_path: Path) -> list[str]:
with zipfile.ZipFile(zip_path, "r") as zf:
return zf.namelist()
+97
View File
@@ -0,0 +1,97 @@
import json
import subprocess
import sys
from pathlib import Path
from .conftest import create_fake_pptx
def test_cli_help_runs_successfully():
repo_root = Path(__file__).resolve().parents[1]
result = subprocess.run(
[sys.executable, "main.py", "--help"],
cwd=repo_root,
capture_output=True,
text=True,
)
assert result.returncode == 0
assert "--input" in result.stdout
assert "--recursive" in result.stdout
assert "--min-kb" in result.stdout
def test_cli_format_folder(tmp_path: Path, sample_images):
repo_root = Path(__file__).resolve().parents[1]
input_dir = tmp_path / "input"
output_dir = tmp_path / "output"
temp_dir = tmp_path / "temp"
logs_dir = tmp_path / "logs"
input_dir.mkdir()
pptx_path = input_dir / "folder_mode.pptx"
create_fake_pptx(
pptx_path,
{
# Usamos JPG para asegurar que pase filtros por tamaño
"photo.jpg": sample_images["photo_jpg"].read_bytes(),
},
)
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps(
{
"paths": {
"input_dir": str(input_dir),
"output_dir": str(output_dir),
"temp_dir": str(temp_dir),
},
"filters": {
"min_kb": 1,
"min_width": 0,
"min_height": 0,
},
"logging": {
"level": "INFO",
"log_file": str(logs_dir / "run.log"),
},
}
),
encoding="utf-8",
)
result = subprocess.run(
[
sys.executable,
"main.py",
"--config",
str(config_path),
"--input",
str(pptx_path),
"--output",
str(output_dir),
"--temp-dir",
str(temp_dir),
"--format",
"folder",
"--min-kb",
"1",
"--log-file",
str(logs_dir / "run.log"),
],
cwd=repo_root,
capture_output=True,
text=True,
)
assert result.returncode == 0, result.stderr
output_folder = output_dir / "folder_mode"
assert output_folder.exists()
assert output_folder.is_dir()
extracted_files = list(output_folder.iterdir())
assert len(extracted_files) == 1
assert (logs_dir / "run.log").exists()
+85
View File
@@ -0,0 +1,85 @@
from pathlib import Path
from document_image_extractor.pipeline import process_file
from .conftest import (create_docx, create_pdf, create_fake_pptx, create_fake_xlsx, zip_names)
def test_docx_extracts_real_extensions(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
docx_path = input_dir / "mixed.docx"
create_docx(docx_path, [sample_images["photo_jpg"], sample_images["big_png"]])
stats = process_file(docx_path, base_cfg)
assert stats["saved"] == 2
output_zip = Path(base_cfg["paths"]["output_dir"]) / "mixed.zip"
assert output_zip.exists()
names = zip_names(output_zip)
assert any(name.endswith(".jpg") for name in names)
assert any(name.endswith(".png") for name in names)
def test_pdf_extracts_image(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
pdf_path = input_dir / "sample.pdf"
create_pdf(pdf_path, sample_images["big_png"])
stats = process_file(pdf_path, base_cfg)
assert stats["found"] >= 1
assert stats["saved"] >= 1
output_zip = Path(base_cfg["paths"]["output_dir"]) / "sample.zip"
assert output_zip.exists()
def test_pptx_extracts_media_images(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
pptx_path = input_dir / "slides.pptx"
create_fake_pptx(
pptx_path,
{
"image1.png": sample_images["big_png"].read_bytes(),
"image2.jpg": sample_images["photo_jpg"].read_bytes(),
},
)
stats = process_file(pptx_path, base_cfg)
assert stats["found"] == 2
assert stats["saved"] == 2
output_zip = Path(base_cfg["paths"]["output_dir"]) / "slides.zip"
assert output_zip.exists()
names = zip_names(output_zip)
assert any(name.endswith(".png") for name in names)
assert any(name.endswith(".jpg") for name in names)
def test_xlsx_extracts_media_images(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
xlsx_path = input_dir / "book.xlsx"
create_fake_xlsx(
xlsx_path,
{
"image1.png": sample_images["big_png"].read_bytes(),
"image2.jpg": sample_images["photo_jpg"].read_bytes(),
},
)
stats = process_file(xlsx_path, base_cfg)
assert stats["found"] == 2
assert stats["saved"] == 2
output_zip = Path(base_cfg["paths"]["output_dir"]) / "book.zip"
assert output_zip.exists()
+137
View File
@@ -0,0 +1,137 @@
from pathlib import Path
from document_image_extractor.pipeline import list_input_files, process_file
from .conftest import create_empty_ooxml, create_fake_pptx, zip_names
def test_dedup_removes_duplicate_media(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
pptx_path = input_dir / "duplicates.pptx"
duplicate_data = sample_images["big_png"].read_bytes()
create_fake_pptx(
pptx_path,
{
"image1.png": duplicate_data,
"image2.png": duplicate_data,
},
)
stats = process_file(pptx_path, base_cfg)
assert stats["found"] == 2
assert stats["saved"] == 1
assert stats["duplicates"] == 1
output_zip = Path(base_cfg["paths"]["output_dir"]) / "duplicates.zip"
assert output_zip.exists()
assert len(zip_names(output_zip)) == 1
def test_dimension_filter_removes_small_icons(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
base_cfg["filters"]["min_kb"] = 1
base_cfg["filters"]["min_width"] = 100
base_cfg["filters"]["min_height"] = 100
pptx_path = input_dir / "filters.pptx"
create_fake_pptx(
pptx_path,
{
"big.png": sample_images["big_png"].read_bytes(),
"icon.png": sample_images["small_icon"].read_bytes(),
},
)
stats = process_file(pptx_path, base_cfg)
assert stats["found"] == 2
assert stats["saved"] == 1
assert stats["filtered_dims"] == 1
output_zip = Path(base_cfg["paths"]["output_dir"]) / "filters.zip"
assert output_zip.exists()
names = zip_names(output_zip)
assert len(names) == 1
def test_empty_ooxml_does_not_create_zip(tmp_path: Path, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
pptx_path = input_dir / "empty.pptx"
create_empty_ooxml(pptx_path)
stats = process_file(pptx_path, base_cfg)
assert stats["found"] == 0
assert stats["saved"] == 0
output_zip = Path(base_cfg["paths"]["output_dir"]) / "empty.zip"
assert not output_zip.exists()
def test_corrupt_file_returns_error_and_cleans_temp(tmp_path: Path, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
corrupt_path = input_dir / "corrupt.xlsx"
corrupt_path.write_bytes(b"this is not a valid zip file")
stats = process_file(corrupt_path, base_cfg)
assert "error" in stats
temp_folder = Path(base_cfg["paths"]["temp_dir"]) / "corrupt"
assert not temp_folder.exists()
def test_list_input_files_recursive(tmp_path: Path, sample_images):
input_dir = tmp_path / "input"
subfolder = input_dir / "subfolder"
subfolder.mkdir(parents=True)
pptx_path = subfolder / "nested.pptx"
create_fake_pptx(
pptx_path,
{
"image1.png": sample_images["big_png"].read_bytes(),
},
)
non_recursive = list_input_files(input_dir, recursive=False)
recursive = list_input_files(input_dir, recursive=True)
assert non_recursive == []
assert recursive == [pptx_path]
def test_format_folder_creates_folder_output(tmp_path: Path, sample_images, base_cfg):
input_dir = tmp_path / "input"
input_dir.mkdir()
base_cfg["output"]["format"] = "folder"
pptx_path = input_dir / "folder_output.pptx"
create_fake_pptx(
pptx_path,
{
"image1.png": sample_images["big_png"].read_bytes(),
},
)
stats = process_file(pptx_path, base_cfg)
assert stats["saved"] == 1
output_folder = Path(base_cfg["paths"]["output_dir"]) / "folder_output"
assert output_folder.exists()
assert output_folder.is_dir()
extracted_files = list(output_folder.iterdir())
assert len(extracted_files) == 1