Difference between revisions of "Build your ZIM file"

Jump to navigation Jump to search
added Zimit and wget-2zim to the "User" section
(added Zimit and wget-2zim to the "User" section)
(12 intermediate revisions by 3 users not shown)
Line 28: Line 28:


=== Create a ZIM file from existing HTML contents ===
=== Create a ZIM file from existing HTML contents ===
See http://www.openzim.org/wiki/Zimwriterfs_instructions for an overview and read the section below on zimwriterfs for some additional context.
See [[Zimwriterfs instructions]] for an overview and read the section below on zimwriterfs for some additional context.
 
=== wget-2-zim ===
[https://github.com/ballerburg9005/wget-2-zim wget-2-zim] is a simple bash script with some nifty tricks that can be used to archive websites on the internet.
 
=== Zimit ===
[https://github.com/openzim/zimit Zimit] ([https://youzim.it online version]) is a scraper allowing to create ZIM file from any website. ''As of 2022 the ZIM files it produces are incompatible with kiwix-desktop, because the Web ARChive implementation it uses require ServiceWorkers (only kiwix-android, kiwix-serve).''


== Developers ==
== Developers ==
Line 35: Line 41:
=== MWoffliner ===
=== MWoffliner ===


MWoffliner is a tool which allows to "dump" a Wikimedia project (Wikipedia, Wiktionary, ...) to a local storage. It should also work for any Mediawiki instance having parsoid installed. It goes through all articles (or a selection if specified) of the project and write HTML/pictures to your local filesystem.
MWoffliner is a tool which allows to "dump" a Wikimedia project (Wikipedia, Wiktionary, ...) to a local storage. It should also work for any [https://mediawiki.org Mediawiki] instance. It goes through all articles (or a selection if specified) of the project and write HTML/pictures to your local filesystem as plain HTML/JS/CSS/... files or in a ZIM file.


A virtual machine with MWoffliner is provided [http://download.kiwix.org/dev/ZIMmaker.ova here]. You might have to update the source code to get the last improvements.
It is distributed via [https://www.npmjs.com/package/mwoffliner npm] and [https://hub.docker.com/r/openzim/mwoffliner Docker].


More information are available [https://github.com/kiwix/mwoffliner here].
If you are a developer, you can download it directly from its [https://github.com/openzim/mwoffliner git repository].


=== zimwriterfs ===
=== zimwriterfs ===
zimwriterfs is a console tool to create ZIM files from a localy stored directory containing a "self-sufficient" HTML content (with pictures, javascript, stylesheets). The result will contain all the files of the local directory compressed and merged in the ZIM file. Nothing more, nothing less. For now, zimwriterfs only works on POSIX compatible systems. You simply need to compile it and run it. The software does not need a lot of resources, but if you create a pretty big ZIM files, then it could take a while to complete.  
zimwriterfs is a console tool to create ZIM files from a localy stored directory containing a "self-sufficient" HTML content (with pictures, javascript, stylesheets). The result will contain all the files of the local directory compressed and merged in the ZIM file. Nothing more, nothing less. For now, zimwriterfs only works on POSIX compatible systems. You simply need to compile it and run it. The software does not need a lot of resources, but if you create a pretty big ZIM files, then it could take a while to complete.  
Instructions on how to prepare and use zimwriterfs are here [[zimwriterfs_instructions]]
Instructions on how to prepare and use zimwriterfs are here [[zimwriterfs_instructions]]
[https://github.com/wikimedia/openzim/tree/master/zimwriterfs Go to zimwriterfs source code repository].
[https://github.com/openzim/zimwriterfs Go to zimwriterfs source code repository].


A virtual machine with zimwriterfs is provided [http://download.kiwix.org/dev/ZIMmaker.ova here].
A virtual machine with zimwriterfs is provided [http://download.kiwix.org/dev/ZIMmaker.ova here].
Line 78: Line 84:


===zimmer===
===zimmer===
The [https://github.com/vss-devel/zimmer zimmer] package allows creating a ZIM dump from any accessible Mediawiki-based wiki. This package is relativly easy to install and supports both old and new versions of Mediawiki.
The [https://github.com/vss-devel/zimmer zimmer] package allows creating a ZIM dump from a Mediawiki-based wiki. This package is relatively easy to install and supports both old and new versions of Mediawiki. It is a kind of an alternative to MWoffliner.


The package consists of two Node.js scripts:
The package consists of two Node.js scripts:
* ''wikizimmer.js'' -- creates static HTML files from the wiki's articles
* ''wikizimmer.js'' -- creates static HTML files from the wiki's articles. It requires public access both to the normal web interface and to the wiki's API interface. Unlike mwoffliner, this script does not require Redis
** Unlike mwoffliner, this script does not require Parsoid or Redis.
* ''zimmer.js'' -- creates a ZIM file from the static HTML files (without requiring the libzim).
* ''zimmer.js'' -- creates a ZIM file from the static HTML files
** Unlike zimwriterfs, this script does not require zimlib.
** This script is used in other projects as a drop-in replacement for zimwriterfs where ''withFullTextIndex'' support is not needed.


== See also ==
== See also ==
* [[ZIM File Archive]]
* [[Bindings]]
* [[Bindings]]
* [[Reader]]
* [[Readers]]

Navigation menu