Difference between revisions of "Docker"

From Bebot Wiki 2
Jump to navigationJump to search
 
(85 intermediate revisions by the same user not shown)
Line 1: Line 1:
===== Docker =====
+
<pre>
 +
                    ##        .       
 +
              ## ## ##      ==        
 +
          ## ## ## ##      ===        
 +
      /""""""""""""""""\___/ ===     
 +
  ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
 +
      \______ o          /           
 +
        \    \        __/           
 +
          \____\______/         
 +
</pre>
 +
===== Preamble =====
 +
 
 +
So using Docker can save you some installation hassle, especially if you're NOT a PHP/SQL expert it will save you huge time ... but beware that understanding Docker also requires some practice of its own !
 +
 
 +
 
 +
Generally, you could consider Docker is no less than amazing if you'd run only one (or very few) bot(s), same if you'd want a very quick discovery on Bebot's functions. In such case Docker is unbeatable on many aspects :
 +
 
 +
With very few actions from you, thanks to Docker's automations based on simple lightweight textfiles, you would configure then install and quickly run some sort of isolated "VMs" bringing your bot(s) and DataBase to life.
 +
 
 +
 
 +
But beyond, eg if you wanted to optimizely run multiple bots from an unique PHP-code installation with all pointing towards same Database or needing some proxy (eg Aocp), it'd possibly not be the best choice as each instance would then weight full Docker image(s)/container(s) in RAM/CPU.
 +
 
 +
For such more complicated cases we'd rather point you at the classical way explained with details within both Bebot README and http://wiki.bebot.link/index.php/Installation
 +
 
  
So using Docker can save you some installation hassle, but you'll need to have it installed properly - depending on system hosting your bot(s) :
+
Now you're aware of this all, and if you feel ready for it, let's proceed into Docker below.
 +
 
 +
 
 +
===== Install =====
 +
 
 +
Before anything else, you'll need to have Docker installed properly - depending on system hosting your bot(s) :
  
 
Windows => https://docs.docker.com/desktop/install/windows-install/
 
Windows => https://docs.docker.com/desktop/install/windows-install/
Line 8: Line 36:
  
 
Mac => https://docs.docker.com/desktop/install/mac-install/
 
Mac => https://docs.docker.com/desktop/install/mac-install/
 +
 +
 +
Also notice that we'll often refer to "console" below. This is a text command line prompt we'll often need.
 +
 +
For Linux/Mac it should be available into your shortcuts.
 +
 +
For Windows you'd have to open menu and type "cmd" or "powershell".
 +
 +
 +
Just make sure you're installed and ready to type, then continue to next section.
 +
 +
 +
===== DB prep =====
 +
 +
Now we have to determine how to manage our DataBase. For the SQL there are three possibilites :
 +
 +
SQL1: you already have an external server ; just make sure it's reachable (eg : ping its ip) from your host, and check if its facial port (usually 3306) is correctly opened.
 +
 +
SQL2: you opted for a local service on host ; so you'd usually go for usual SQL port (3306) but instead of 127.0.0.1 default ip we'd advice you default docker0 network ip 172.17.0.1
 +
 +
SQL3: you are about to run a container for that ; you can skip this part fully as we'll explain you how to set this up correctly later in "build" section, it's the simplest in fact !
 +
 +
 +
Here some command or documentation to verify elements for cases SQL1/2 :
 +
 +
- IP/Network on Linux/Mac (try from a console : ip a) and Windows (also in console : ipconfig)
 +
 +
- Ports/Services on Linux/Mac https://vitux.com/find-open-ports-on-debian and Windows https://www.geeksforgeeks.org/how-to-check-open-tcp-ip-ports-in-windows
 +
 +
- also find SQL config (usually .cnf) file(s) and check that your parameter "bind-address" is correct, otherwise change it and restart service
 +
 +
 +
! <span style="color:#CC9900">ATTENTION</span> ! in upper cases SQL1/2 you will have to :
 +
 +
- CREATE a DATABASE dedicated for the bot (you can name it as you like but remember it)
 +
 +
- make an USER with PASSWORD and expected source (eg : @'%' to accept external requests)
 +
 +
- also GRANT that USER all PRIVILEGES on the DATABASE you created upper
 +
 +
 +
If needed, you may have to read some SQL doc, as https://dev.mysql.com/doc/ and adapt it to you context.
 +
 +
 +
But for generic example it will usually look like the following :
 +
 +
mysql (enters prompt, optional parameters are --user --password + for distant server --host)
 +
 +
> CREATE DATABASE bebotdbname;
 +
 +
> CREATE USER 'bebotuzr'@'%' IDENTIFIED BY 'botpass';
 +
 +
> GRANT ALL PRIVILEGES ON bebotdbname.* TO 'bebotuzr'@'%';
 +
 +
 +
If you ain't sure, you can verify your work is fine by doing :
 +
 +
> SHOW DATABASES; (should show the DB you created upper among list)
 +
 +
> SELECT User, Host FROM mysql.user; (should show your upper user among list)
 +
 +
> SHOW GRANTS FOR 'bebotuzr'@'%'; (should show the upper GRANT among list)
 +
 +
 +
If all here is done (or if you're in upper case SQL3) let's move on forward.
 +
  
 
===== Build =====
 
===== Build =====
  
Once the software is installed, you'll have to build a container based on Bebot itself + docker elements.
+
Once the software is installed, you first have to build runnable images from Bebot Docker preset configs :
 +
 
 +
Get them from https://github.com/bitnykk/DockerBeBot/ (download/decompress the .zip or use git clone)
 +
 
 +
 
 +
Then in command line, enter (cd) into the folder corresponding to your situation. Their name have 3 parts :
 +
 
 +
1: "linux" or "windows", which depends on your host Operating System,
 +
 
 +
2: "official" to run latest stable bot, or "sandbox" for some dev/beta testing,
 +
 
 +
3: "botonly" (in upper cases SQL1/2) or "botplusdb" (in upper case SQL3).
 +
 
 +
 
 +
Once inside the folder meeting your requirements, you can check (ls or dir) its content :
 +
 
 +
You get those 3 files : docker-compose.yml | docker-entrypoint.sh | Dockerfile
 +
 
 +
Note : you could edit Dockerfile and change PHP version for whatever you like ; more details on its modules/config at https://www.php.net/docs.php
 +
 
 +
In upper case SQL3, you'd also have to setup your SQL credentials within docker-compose.yml (4 values to modify under "environment" part).
 +
 
 +
 
 +
Once everything seems correct according to your needs, just do :
 +
 
 +
<pre>docker compose build </pre>
 +
 
 +
If all goes fine you should see stuff as "Building X.Ys (10/10) FINISHED" + <span style="color:#0000FF">several blue lines starting with "=>"</span>.
 +
 
 +
 
 +
When it's over you are almost ready for the most exciting part ... but we'll need few more actions first ...
 +
 
 +
 
 +
===== Bot config =====
 +
 
 +
Beware Docker works in a way that makes the bot could wipes some files at reload so we need to render some persistents.
 +
 
 +
For that reason we will extract some files within the image we built earlier, and save them at host level to protect them.
 +
 
 +
To do this, we will now run this long line on Linux from within upper chosen folder to get bindable datas :
 +
 
 +
<pre>
 +
docker run --rm --entrypoint tar bebot-image czf - Conf Custom Extras/Bank Extras/Scripts log Text > out.tar.gz && tar xzf out.tar.gz -C . && rm out.tar.gz && chown 1000:1000 * -R
 +
</pre>
 +
 
 +
On Windows this can't work properly, so we will let Docker manage volumes for us that will soon be explorable at network location \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\
 +
 
 +
https://stackoverflow.com/questions/61083772/where-are-docker-volumes-located-when-running-wsl-using-docker-desktop
 +
 
  
So you can download Bebot code (either from Official/Stable or Sandbox/Dev) as .zip (or using git pull command).
+
In any case we'll get 5 new folders ("Conf" "Custom" "Extras" "log" "Text") in chosen folder - where our 3 Docker config files are also.
  
Once done you'll have a Bebot folder in which you will create 2 text files :
+
Note : in upper case SQL3 we'll also soon have a "db" folder later at the same place, once we'll have our DB container started for first time.
 +
 
 +
Windows only : possibly set permissive rights on those folders/files (select all, right-click > Properties > Security) so our container access them.
 +
 
 +
===== Run =====
 +
 
 +
We're now ready to run our container(s) : 1 only if we want Bebot with our usual SQL server, otherwise 2 for Bebot + SQL both as Docker containers.
  
(no extension)
 
Dockerfile
 
 
<pre>
 
<pre>
FROM alpine:latest
+
docker compose run bebot
ENTRYPOINT ["/sbin/tini", "-g", "--"]
 
CMD ["/bebot/docker-entrypoint.sh"]
 
RUN apk --no-cache --repository http://dl-3.alpinelinux.org/alpine/edge/community/ add \
 
    php82-cli php82-sqlite3 php82-phar php82-curl php82-sockets php82-pdo php82-pdo_sqlite \
 
    php82-pdo_mysql php82-mbstring php82-ctype php82-bcmath php82-json php82-posix php82-simplexml \
 
php82-dom php82-pcntl php82-zip php82-opcache php82-fileinfo php82-mysqli \
 
tini \
 
    sudo \
 
    && \
 
    adduser -h /bebot -s /bin/false -D -H bebot
 
COPY --chown=bebot:bebot . /bebot
 
RUN sudo ln -s /usr/bin/php82 /usr/bin/php8
 
USER bebot
 
WORKDIR /bebot
 
 
</pre>
 
</pre>
  
===== test2 =====
+
Your bot should now load (after launching its DB if some), check its updates then show its ip within Docker network.
...
+
 
 +
As it's our first run, bot will then ask us to fill up credentials informations.
 +
 
 +
So give him account name, pass, character, server and also owner and superadmin(s) plus few more questions.
 +
 
 +
Then you're asked for the SQL part. In upper cases SQL1/2 you should already have everything needed here.
 +
 
 +
For upper case SQL3 you db ip should but the same than bot but end in .2 (as .1 should be host and .3 the bot itself).
 +
 
 +
 
 +
Finish the setting up, then bot will create all his DB entries, and endly come online !
 +
 
 +
NB : expect some deep caching at very first run keeping the bot busy for a minute or two. If anything goes wrong, restart containers (see below).
 +
 
 +
 
 +
===== Control =====
 +
 
 +
Now to check if your container(s) run properly, you have a set of commands :
 +
 
 +
To obtain a list of created image(s) : docker image ls
 +
 
 +
To show a list of running container(s) : docker ps (can use -a parameter to also see stopped ones)
 +
 
 +
To see realtime stats of container(s) : docker stats (Ctrl+c to exit)
 +
 
 +
To passively check realtime log of a container : docker logs -f container_name_or_id (Ctrl+c to exit which will NOT shutdown bot/container)
 +
 
 +
To enter container thread interactively : docker attach container_name_or_id (Ctrl+p Ctrl+q to exit without shutting down)
 +
 
 +
To execute container interactively : docker exec -it container_name_or_id sh (Ctrl+p Ctrl+q for the same result that upper)
 +
 
 +
To copy from a running container : docker cp container_name_or_id:/path/to/stuff.ext .
 +
 
 +
The same towards a running container : docker cp stuff.ext container_name_or_id:/path/to/copy/to
 +
 
 +
 
 +
If you need to stop (and then restart) some container(s) :
 +
 
 +
To stop a given container : docker stop container_name_or_id
 +
 
 +
To stop all containers together : docker stop $(docker ps -q)
 +
 
 +
To restart stopped bot : docker compose run -d bebot (-d so it stays in background)
 +
 
 +
 
 +
If Bebot git code was patched, such container restart upper should quickfix to auto-update it.
 +
 
 +
For best reliability you also can delete bot image (its container once stopped/removed) and rebuild it with --no-cache parameter added :
 +
 
 +
- stop all (docker compose down ; docker stop containername)
 +
 
 +
- find & delete the bot container (docker ps -a ; docker rm fullid)
 +
 
 +
- find & delete the bot image (docker image ls ; docker image rm imagename)
 +
 
 +
- rebuild the bot image (docker compose build servicename --no-cache)
 +
 
 +
- restart the whole network (docker compose run -d servicename)
 +
 
 +
 
 +
You may setup some task(s) (crontab on Linux/Mac / Task Manager > Startup on Windows) to run some container(s) right at host startup.
 +
 
 +
Sources : https://manpages.debian.org/bullseye/cron/crontab.5.en.html / https://www.howtogeek.com/208224/how-to-add-a-program-to-startup-in-windows/
 +
 
 +
Same way you should strongly save your SQL datas frequently as Docker won't do it for you ; can use task(s) based on mysqldump for that.
 +
 
 +
Source : https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html
 +
 
 +
 
 +
===== Maintenance =====
 +
 
 +
Before anything, you can troubleshoot most common mistakes verifying in order :
 +
 
 +
- correct software installation (Docker ahead of any other)
 +
 
 +
- your various elements ip, port, state and availability
 +
 
 +
- you credentials (check bot's Conf folder in all cases, Docker's compose for case SQL3)
 +
 
 +
 
 +
If needed to make house cleaning (assuming you've made <span style="color:#BB0000">BACKUPS</span> of sensible datas you could lose below !) :
 +
 
 +
To stop the containers : docker compose down (possibly --volumes param added)
 +
 
 +
To delete the recent containers : docker rm $(docker ps -a -q)
 +
 
 +
To clear a given image : docker image rm -f image_name
 +
 
 +
To remove all unused images : docker image prune -a
 +
 
 +
To trash all useless networks : docker network prune
 +
 
 +
For all upper cleans + cache : docker system prune -a
 +
 
 +
 
 +
Again, any of the upper could make you lose datas, so cover yourself with some previous <span style="color:#BB0000">BACKUPS</span> (or do not blame your poor computer).
 +
 
 +
 
 +
===== Conclusion =====
 +
 
 +
This guide is a work in progress, kept as simple as possible, and will evolve as we find improvements or tricks.
 +
 
 +
For more details and options you can refer to Docker official documentation here https://docs.docker.com/

Latest revision as of 21:03, 5 December 2024

                    ##        .         
              ## ## ##       ==         
           ## ## ## ##      ===         
       /""""""""""""""""\___/ ===       
  ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
       \______ o          /             
         \    \        __/             
          \____\______/           
Preamble

So using Docker can save you some installation hassle, especially if you're NOT a PHP/SQL expert it will save you huge time ... but beware that understanding Docker also requires some practice of its own !


Generally, you could consider Docker is no less than amazing if you'd run only one (or very few) bot(s), same if you'd want a very quick discovery on Bebot's functions. In such case Docker is unbeatable on many aspects :

With very few actions from you, thanks to Docker's automations based on simple lightweight textfiles, you would configure then install and quickly run some sort of isolated "VMs" bringing your bot(s) and DataBase to life.


But beyond, eg if you wanted to optimizely run multiple bots from an unique PHP-code installation with all pointing towards same Database or needing some proxy (eg Aocp), it'd possibly not be the best choice as each instance would then weight full Docker image(s)/container(s) in RAM/CPU.

For such more complicated cases we'd rather point you at the classical way explained with details within both Bebot README and http://wiki.bebot.link/index.php/Installation


Now you're aware of this all, and if you feel ready for it, let's proceed into Docker below.


Install

Before anything else, you'll need to have Docker installed properly - depending on system hosting your bot(s) :

Windows => https://docs.docker.com/desktop/install/windows-install/

Linux => https://docs.docker.com/desktop/install/linux-install/

Mac => https://docs.docker.com/desktop/install/mac-install/


Also notice that we'll often refer to "console" below. This is a text command line prompt we'll often need.

For Linux/Mac it should be available into your shortcuts.

For Windows you'd have to open menu and type "cmd" or "powershell".


Just make sure you're installed and ready to type, then continue to next section.


DB prep

Now we have to determine how to manage our DataBase. For the SQL there are three possibilites :

SQL1: you already have an external server ; just make sure it's reachable (eg : ping its ip) from your host, and check if its facial port (usually 3306) is correctly opened.

SQL2: you opted for a local service on host ; so you'd usually go for usual SQL port (3306) but instead of 127.0.0.1 default ip we'd advice you default docker0 network ip 172.17.0.1

SQL3: you are about to run a container for that ; you can skip this part fully as we'll explain you how to set this up correctly later in "build" section, it's the simplest in fact !


Here some command or documentation to verify elements for cases SQL1/2 :

- IP/Network on Linux/Mac (try from a console : ip a) and Windows (also in console : ipconfig)

- Ports/Services on Linux/Mac https://vitux.com/find-open-ports-on-debian and Windows https://www.geeksforgeeks.org/how-to-check-open-tcp-ip-ports-in-windows

- also find SQL config (usually .cnf) file(s) and check that your parameter "bind-address" is correct, otherwise change it and restart service


! ATTENTION ! in upper cases SQL1/2 you will have to :

- CREATE a DATABASE dedicated for the bot (you can name it as you like but remember it)

- make an USER with PASSWORD and expected source (eg : @'%' to accept external requests)

- also GRANT that USER all PRIVILEGES on the DATABASE you created upper


If needed, you may have to read some SQL doc, as https://dev.mysql.com/doc/ and adapt it to you context.


But for generic example it will usually look like the following :

mysql (enters prompt, optional parameters are --user --password + for distant server --host)

> CREATE DATABASE bebotdbname;

> CREATE USER 'bebotuzr'@'%' IDENTIFIED BY 'botpass';

> GRANT ALL PRIVILEGES ON bebotdbname.* TO 'bebotuzr'@'%';


If you ain't sure, you can verify your work is fine by doing :

> SHOW DATABASES; (should show the DB you created upper among list)

> SELECT User, Host FROM mysql.user; (should show your upper user among list)

> SHOW GRANTS FOR 'bebotuzr'@'%'; (should show the upper GRANT among list)


If all here is done (or if you're in upper case SQL3) let's move on forward.


Build

Once the software is installed, you first have to build runnable images from Bebot Docker preset configs :

Get them from https://github.com/bitnykk/DockerBeBot/ (download/decompress the .zip or use git clone)


Then in command line, enter (cd) into the folder corresponding to your situation. Their name have 3 parts :

1: "linux" or "windows", which depends on your host Operating System,

2: "official" to run latest stable bot, or "sandbox" for some dev/beta testing,

3: "botonly" (in upper cases SQL1/2) or "botplusdb" (in upper case SQL3).


Once inside the folder meeting your requirements, you can check (ls or dir) its content :

You get those 3 files : docker-compose.yml | docker-entrypoint.sh | Dockerfile

Note : you could edit Dockerfile and change PHP version for whatever you like ; more details on its modules/config at https://www.php.net/docs.php

In upper case SQL3, you'd also have to setup your SQL credentials within docker-compose.yml (4 values to modify under "environment" part).


Once everything seems correct according to your needs, just do :

docker compose build 

If all goes fine you should see stuff as "Building X.Ys (10/10) FINISHED" + several blue lines starting with "=>".


When it's over you are almost ready for the most exciting part ... but we'll need few more actions first ...


Bot config

Beware Docker works in a way that makes the bot could wipes some files at reload so we need to render some persistents.

For that reason we will extract some files within the image we built earlier, and save them at host level to protect them.

To do this, we will now run this long line on Linux from within upper chosen folder to get bindable datas :

docker run --rm --entrypoint tar bebot-image czf - Conf Custom Extras/Bank Extras/Scripts log Text > out.tar.gz && tar xzf out.tar.gz -C . && rm out.tar.gz && chown 1000:1000 * -R

On Windows this can't work properly, so we will let Docker manage volumes for us that will soon be explorable at network location \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\

https://stackoverflow.com/questions/61083772/where-are-docker-volumes-located-when-running-wsl-using-docker-desktop


In any case we'll get 5 new folders ("Conf" "Custom" "Extras" "log" "Text") in chosen folder - where our 3 Docker config files are also.

Note : in upper case SQL3 we'll also soon have a "db" folder later at the same place, once we'll have our DB container started for first time.

Windows only : possibly set permissive rights on those folders/files (select all, right-click > Properties > Security) so our container access them.

Run

We're now ready to run our container(s) : 1 only if we want Bebot with our usual SQL server, otherwise 2 for Bebot + SQL both as Docker containers.

docker compose run bebot

Your bot should now load (after launching its DB if some), check its updates then show its ip within Docker network.

As it's our first run, bot will then ask us to fill up credentials informations.

So give him account name, pass, character, server and also owner and superadmin(s) plus few more questions.

Then you're asked for the SQL part. In upper cases SQL1/2 you should already have everything needed here.

For upper case SQL3 you db ip should but the same than bot but end in .2 (as .1 should be host and .3 the bot itself).


Finish the setting up, then bot will create all his DB entries, and endly come online !

NB : expect some deep caching at very first run keeping the bot busy for a minute or two. If anything goes wrong, restart containers (see below).


Control

Now to check if your container(s) run properly, you have a set of commands :

To obtain a list of created image(s) : docker image ls

To show a list of running container(s) : docker ps (can use -a parameter to also see stopped ones)

To see realtime stats of container(s) : docker stats (Ctrl+c to exit)

To passively check realtime log of a container : docker logs -f container_name_or_id (Ctrl+c to exit which will NOT shutdown bot/container)

To enter container thread interactively : docker attach container_name_or_id (Ctrl+p Ctrl+q to exit without shutting down)

To execute container interactively : docker exec -it container_name_or_id sh (Ctrl+p Ctrl+q for the same result that upper)

To copy from a running container : docker cp container_name_or_id:/path/to/stuff.ext .

The same towards a running container : docker cp stuff.ext container_name_or_id:/path/to/copy/to


If you need to stop (and then restart) some container(s) :

To stop a given container : docker stop container_name_or_id

To stop all containers together : docker stop $(docker ps -q)

To restart stopped bot : docker compose run -d bebot (-d so it stays in background)


If Bebot git code was patched, such container restart upper should quickfix to auto-update it.

For best reliability you also can delete bot image (its container once stopped/removed) and rebuild it with --no-cache parameter added :

- stop all (docker compose down ; docker stop containername)

- find & delete the bot container (docker ps -a ; docker rm fullid)

- find & delete the bot image (docker image ls ; docker image rm imagename)

- rebuild the bot image (docker compose build servicename --no-cache)

- restart the whole network (docker compose run -d servicename)


You may setup some task(s) (crontab on Linux/Mac / Task Manager > Startup on Windows) to run some container(s) right at host startup.

Sources : https://manpages.debian.org/bullseye/cron/crontab.5.en.html / https://www.howtogeek.com/208224/how-to-add-a-program-to-startup-in-windows/

Same way you should strongly save your SQL datas frequently as Docker won't do it for you ; can use task(s) based on mysqldump for that.

Source : https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html


Maintenance

Before anything, you can troubleshoot most common mistakes verifying in order :

- correct software installation (Docker ahead of any other)

- your various elements ip, port, state and availability

- you credentials (check bot's Conf folder in all cases, Docker's compose for case SQL3)


If needed to make house cleaning (assuming you've made BACKUPS of sensible datas you could lose below !) :

To stop the containers : docker compose down (possibly --volumes param added)

To delete the recent containers : docker rm $(docker ps -a -q)

To clear a given image : docker image rm -f image_name

To remove all unused images : docker image prune -a

To trash all useless networks : docker network prune

For all upper cleans + cache : docker system prune -a


Again, any of the upper could make you lose datas, so cover yourself with some previous BACKUPS (or do not blame your poor computer).


Conclusion

This guide is a work in progress, kept as simple as possible, and will evolve as we find improvements or tricks.

For more details and options you can refer to Docker official documentation here https://docs.docker.com/