Difference between revisions of "Zimwriterfs instructions"

From openZIM
Jump to navigation Jump to search
(→‎Preparations: Added new section on using Docker)
(I added a paragraph about how to archive websites with this tool, my previous edit was not saved properly.)
(7 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= zimwriterfs instructions =
= zimwriterfs instructions =
Here are some notes on how to prepare your materials and use `zimwriterfs`. The notes are incomplete as they're based on my limited experience using the tool in the prebuilt Docker Image. The Docker file is available at https://hub.docker.com/r/openzim/zimwriterfs/
Here are some notes on how to prepare your materials and use `zimwriterfs`.
These instructions are a community contribution, provided as-is by users of the tool.
 
== Preparations ==
== Preparations ==
Some considerations can help you prepare your materials
 
`zimwriterfs` is a tool which will pack all files (HTML/JS/CSS/JPEG/WEBM/...) belonging to a directory in a ZIM file.
 
Before using this tool, you must hence ensure you have a self-sufficient directory of all content you want to pack.
 
Some considerations can help you prepare your materials:


# Can it be read and navigated in a mainstream web browser e.g. using <code>file://localhost/mycontent</code> (replace mycontent with the correct URL)
# Can it be read and navigated in a mainstream web browser e.g. using <code>file://localhost/mycontent</code> (replace mycontent with the correct URL)
# Would you like to use the classic ZIM file internal structures? e.g. <code>./-/</code> for CSS and <code>./A/</code> for articles (in HTML format), etc.? If so, there's no need to use the <code>--uniqueNamespace</code> command-line option. Otherwise please use it.
# Would you like to use the recommended ZIM file internal structures? e.g. <code>./-/</code> for CSS and <code>./A/</code> for articles (in HTML format), etc.? If so, there's no need to use the <code>--uniqueNamespace</code> command-line option. Otherwise you should use it.
# Have you identified your welcome page and got a suitable icon?
# Have you identified your welcome page and got a suitable icon?
# Have you created a shared location available to the running Docker image to make the content easier to access by zimwriterfs?[1]
# Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.
# Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.


[1] I added a shared folder so I could easily share data between the running Docker image and my macbook. This had to be configured using the command line. See https://github.com/rocker-org/rocker/wiki/Sharing-files-with-host-machine for tips on how to do this.
=== Running zimwritefs with Docker ===


=== Configuring and Running Docker
The simplest setup to run `zimwriterfs` on a variety of configuration is to use Docker since it will ensure a smooth operation no matter which OS your machine is running.
Again some qualifications:


* I've limited experience in using Docker.
As `zimwriterfs` is now part of `zim-tools`, you have to use the `openzim/zim-tools` image. And of course you need a functional installation of Docker (including a proper subscription, adapted to new Docker licencing model - it is free if you use it for personal use)
* This worked for me but might need tweaking for others.
* I use Kitematic (available in Docker) to obtain a shell in the running Docker image.


Here's how to share the current folder with the Docker image. The shared folder is at <code>/Volumes/shared</code> in the running Docker image.
For simplicity, we will share a directory with the Docker container:
- create a root folder for your generated zim file + content
- put every content in the `dist` subfolder
- move into the root folder you created
- you can now start zimwriterfs with a command like the following one. With this command, the shared folder is at <code>/data</code> in the running Docker image.
<code>
<code>
docker run -it -v $(pwd):/Volumes/shared -p 8888:8888 openzim/zimwriterfs
docker run -it -v $(pwd):/data/dist openzim/zimwriterfs zimwriterfs ...
</code>
</code>
=== Running zimwritefs in a script with wget to archive websites ===
[https://github.com/openzim/zimit Zimit] uses the WebARChive standard, which is preferable over wget. However as of 2022 the implementation is incompatible with Kiwix-Desktop. You might want to have a look at [https://github.com/ballerburg9005/wget-2-zim wget-2-zim].
== Examples of using zimwriterfs ==


<code>
<code>
zimwriterfs  --uniqueNamespace  -w index.html -f icon.png -l EN -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /Volumes/shared/dist/ /Volumes/shared/th-ln.zim
docker run -it -v $(pwd):/data openzim/zim-tools zimwriterfs -w index.html -I icon.png -l eng -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /data/dist /data/th-ln.zim
 
docker run -it -v $(pwd):/data openzim/zim-tools zimwriterfs -w index.html -I tess.png -l eng -t "Visions for Teaching and Learning" -d "Sample TESS content: Week 1" -c "Open University" -p "Julian Harty" /data/dist /data/week1.zim
</code>
</code>



Revision as of 18:27, 3 May 2022

zimwriterfs instructions

Here are some notes on how to prepare your materials and use `zimwriterfs`. These instructions are a community contribution, provided as-is by users of the tool.

Preparations

`zimwriterfs` is a tool which will pack all files (HTML/JS/CSS/JPEG/WEBM/...) belonging to a directory in a ZIM file.

Before using this tool, you must hence ensure you have a self-sufficient directory of all content you want to pack.

Some considerations can help you prepare your materials:

  1. Can it be read and navigated in a mainstream web browser e.g. using file://localhost/mycontent (replace mycontent with the correct URL)
  2. Would you like to use the recommended ZIM file internal structures? e.g. ./-/ for CSS and ./A/ for articles (in HTML format), etc.? If so, there's no need to use the --uniqueNamespace command-line option. Otherwise you should use it.
  3. Have you identified your welcome page and got a suitable icon?
  4. Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.

Running zimwritefs with Docker

The simplest setup to run `zimwriterfs` on a variety of configuration is to use Docker since it will ensure a smooth operation no matter which OS your machine is running.

As `zimwriterfs` is now part of `zim-tools`, you have to use the `openzim/zim-tools` image. And of course you need a functional installation of Docker (including a proper subscription, adapted to new Docker licencing model - it is free if you use it for personal use)

For simplicity, we will share a directory with the Docker container: - create a root folder for your generated zim file + content - put every content in the `dist` subfolder - move into the root folder you created - you can now start zimwriterfs with a command like the following one. With this command, the shared folder is at /data in the running Docker image. docker run -it -v $(pwd):/data/dist openzim/zimwriterfs zimwriterfs ...

Running zimwritefs in a script with wget to archive websites

Zimit uses the WebARChive standard, which is preferable over wget. However as of 2022 the implementation is incompatible with Kiwix-Desktop. You might want to have a look at wget-2-zim.

Examples of using zimwriterfs

docker run -it -v $(pwd):/data openzim/zim-tools zimwriterfs -w index.html -I icon.png -l eng -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /data/dist /data/th-ln.zim

docker run -it -v $(pwd):/data openzim/zim-tools zimwriterfs -w index.html -I tess.png -l eng -t "Visions for Teaching and Learning" -d "Sample TESS content: Week 1" -c "Open University" -p "Julian Harty" /data/dist /data/week1.zim

Test the ZIM locally

Use at least one reader e.g. a Kiwix app or server to test the ZIM file contains the expected contents. The Welcome page is displayed, CSS is applied, Images appear, etc.

Known issues

Symlinks to content cause zimwriterfs to abort https://github.com/openzim/libzim/issues/16