34 lines
1.5 KiB
Plaintext
34 lines
1.5 KiB
Plaintext
---
|
|
id: export-entire-dataset-to-file
|
|
title: Export entire dataset to file
|
|
---
|
|
|
|
import ApiLink from '@site/src/components/ApiLink';
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
|
|
|
|
import JsonExample from '!!raw-loader!roa-loader!./code_examples/export_entire_dataset_to_file_json.py';
|
|
import CsvExample from '!!raw-loader!roa-loader!./code_examples/export_entire_dataset_to_file_csv.py';
|
|
|
|
This example demonstrates how to use the <ApiLink to="class/BasicCrawler#export_data">`BasicCrawler.export_data`</ApiLink> method of the crawler to export the entire default dataset to a single file. This method supports exporting data in either CSV or JSON format and also accepts additional keyword arguments so you can fine-tune the underlying `json.dump` or `csv.writer` behavior.
|
|
|
|
:::note
|
|
|
|
For these examples, we are using the <ApiLink to="class/BeautifulSoupCrawler">`BeautifulSoupCrawler`</ApiLink>. However, the same method is available for other crawlers as well. You can use it in exactly the same way.
|
|
|
|
:::
|
|
|
|
<Tabs groupId="main">
|
|
<TabItem value="json" label="JSON">
|
|
<RunnableCodeBlock className="language-python" language="python">
|
|
{JsonExample}
|
|
</RunnableCodeBlock>
|
|
</TabItem>
|
|
<TabItem value="csv" label="CSV">
|
|
<RunnableCodeBlock className="language-python" language="python">
|
|
{CsvExample}
|
|
</RunnableCodeBlock>
|
|
</TabItem>
|
|
</Tabs>
|