"Backup" and "Full Backups"
The farmerswife (fw) Server application comes with two different built-in backup solutions, which complement each other:
- "Backup" or "DB Save" process
- "Full Backup" functionality
The "Backup" or "DB Save" process:
The "Backup" or "DB Save" process continuously takes timed snapshot backups of your main farmerswife database (current45.efdb) and the SQL-lite helper-databases (fwdb.db3 and despatches.db3) files.
On the fw Server app's Log window or within the log.txt files this is the info you should always see:
PID: yyyymmdd:hhmm: DB Save: Starting PID: yyyymmdd:hhmm: DB Save: Saving To Disk PID: yyyymmdd:hhmm: DB Save: Saving Sqlite DB To Disk PID: yyyymmdd:hhmm: DB Save: Saving Backup PID: yyyymmdd:hhmm: DB Save Completed
PID = the fw Server app's running Process-ID
yyyymmdd:hhmm = 2023-01-01:23:42
How it works:
On the first launch of the fw Server app, it will check for existing DB files within the "system" folder.
If it can't find any, it will offer to launch with using the "DemoDB" or "EmptyDB".
On the first 3 steps of the DB Save process, the data for the current45.efdb, despatches.db3 and fwdb.db3 get flushed down from memory into temporary files within the "system" folder, which contain this pre-fix on each file: tmp_db_idyyyy...
On the fourth step - Saving Backup - the "previous" current45.efdb, despatches.db3 and fwdb.db3 get renamed and moved from the "system" folder to the "backups" sub-folder.
And once the "tmp_db_idyyyy..." files are renamed again to current45.efdb, despatches.db3 and fwdb.db3, the whole DB Save process is done with this step: DB Save Completed.
These are stored within the "backups" sub-folder within the farmerswife Server's "system" folder.
"Backup" configuration:
We recommend for the timed interval of the "Backup Every:" setting to be set to: 15 min.!
And the corresponding "Backups" setting should either be set to "Auto" or "100" to cover 24h.
If you deviate from this pattern ensure your hardware / free disk space is up for the task in relation to the sizes of the involved files!
Also NOTE: the intervals should never be longer than 15 min. to minimize potential data loss on a catastrophic event; e.g. a power-cut on the fw Server host machine.
The fw Server app is allowed to skip the "DB Save" process once! And should the "DB Save" process fail again, then the fw Server switches into "panic mode" and will attempt this right away multiple times and finally kill itself and crash to get your attention that something is wrong! In this scenario and above recommended configuration the max. duration of data loss is for the last 30 min.
Backup Every: Choose how often the fw Server will do a "DB Save" process; recommended best practice and default is set to Backup Every: 15 min.
Backups: Select the amount of Backups to be "kept" and thus stored within the "backups" folder, i.e. once that number is reached the oldest backup will be deleted.
Note: if you do a mouse over the word "Backups" you will see the behavior of the "Auto" option. It will display similar info to this:
Backups: This setting sets the maximum number of backups to be stored.
The new "Auto" setting automatically saves backups according to the following schedule:
Example assuming "Backup Every" is set to 15 min.
Once per 15 minutes for the past 2 hours.
Details:
From 13:00 to 15:00 results in 8 backups.
Once per hour for the past day.
Details:
From 13:00 to 13:00 the next day results in 8 backups + 1 per hour x 22 (because the latest 2 hours are already covered).
8 + 22 = 30 backups.
Once per day for the past 2 weeks.
Details:
8 + 22 + 13 = 43 backups.
For the next 13 days, because 1 day is covered.
Once per week for the past 10 weeks.
Details:
8 + 22 + 13 + 8 = 51 backups.
For the next 8 weeks, because the latest 2 weeks are already covered.
Once per month for the past year
Details:
8 + 22 + 13 + 8 + 10 = 61
For the next 8 weeks, because the latest 2 weeks are already covered.
IMPORTANT: there will be a couple of more Backups, because the algorithm always takes the safer path.
Based on your settings about ... backups will be stored.
And the estimated total storage size based on the size of your current backups: ... MB
The "Full Backup":
The "Full Backup" functionality is similar, but this is designed for "daily system backups". This process copies the fw Server's root installation directory/package, and by default does NOT include the "files" sub folder, which typically can contain big video clips.
VERY IMPORTANT: The "Full Backup Folder" path should NOT be pointing to the same disk/volume as the fw Server's root-installation folder!
VERY IMPORTANT: Ensure that you provide enough storage capacity depending on the type and quantity of files you will be and are using within farmerswife. This should be verified at least once per year.
Full Backup Time: Select at what time the farmerswife Server shall perform the Full Backup. The Server application must be running at that time.
Full Backups: Select the number of Full Backups to be stored, i.e. once that number is reached the oldest full backup will be deleted.
Zip Backups: By default, this legacy setting is disabled in order to improve the stability and performance of your system. Do NOT enable it! This is a legacy setting, which can cause random low-level crashes. Also, this setting will be removed in the foreseeable future.
Full Backup Folder: Click on the path selector field, to point to a specific network storage path for the "Full Backup" folder. We recommend creating a new backup-folder on a different physical hard drive, as a sensible precaution against data loss. Please make sure that the Server application has access to this folder and that the path is correctly defined. On Windows, this will need to be correctly "mounted" with a Drive Letter. Also ensure regardless of the used Operating System, that this path will be auto-mounted after a host machine restart.
Include Files Folder In Full Backup: Select "Yes" to include all files and clips in your "Full Backup" folder. This setting together with the amount of data you have stored within the "files" folder has direct implications on how long the Full Backup process will take. If folders have been broken out using the various built-in functionality, this will prevent this feature from being enabled. Here we're typically talking about Terabytes of data, being located on network storage locations. Here, other backup solutions need to be put in place on your side.
Forced Shutdown:
This functionality provides automated nightly restarts of your WIFE system. Select Yes if you wish to restart the Server during the night.
Mode: By default, it is set to "Restart". But if you are running the farmerswife Server as a service of your operating system, select "Quit" and define your desired settings within your operating system.
Time: Select the time when the defined action should take place.
Weekdays: This functionality was implemented on v6.4 and is intended to be used together with "Use SQL" and "SQL Mode > Full Dump Only". This might be changed on the following versions.
Manual Backup Operations:
- Backup Now: By clicking this button on the running fw Server application window you manually initiate the standard database back-up process. In the Server Log Window, you will now see “Manual Database Save” followed by the 5-step "DB Save" process.
- Full Backup Now: Click this button on the running fw Server application window to create a manual "Full Backup" of the farmerswife Server application. This operation is recommended to be done before an upgrade of the fw Server application or any other modifications within the installation folder structure. The Full Backup path can be set to the local machine or any host which is accessible through the LAN. We recommend to NOT store the "Full Backups" on the same volume/disk as where the fw Server application is installed!
How to restore a Backup:
To restore a backup, shut down (Quit) the fw Server application (notify all connected users first), and access the application installation directory.
On Mac typically go to your "Applications" folder and on the "farmerswife Server" app select it and right-mouse click and use "Show Package Contents". You should now see the "system" folder.
On Windows you can typically use the "farmerswife Server" Desktop icon, right-mouse click and use "Open file location" and the Windows Explorer will open inside the fw Server's root installation folder. You should now see the "system" folder.
- Go to the "system" folder and first review the "log.txt", to get a better understanding of what might have happened, and how far back in time you need to go.
The oldest data is at the top and the latest data is at the bottom of the log.txt file.
Scroll to the very end of the log.txt file and then gently scroll up.
"DB Save" complete process:
Process ID: Date-Stamp:TimeStamp: Action
1234: yyyymmdd:hhmm: DB Save Starting1234: yyyymmdd:hhmm: DB Save: Saving To Disk
1234: yyyymmdd:hhmm: DB Save: Saving Sqlite DB To Disk
1234: yyyymmdd:hhmm: DB Save: Saving Backup
1234: yyyymmdd:hhmm: DB Save Completed
"INIT"
This indicates a fw Server app launch.
You might have to repeat and try different sets of DB files by going back in time. This depends on the nature of the incident. - Then re-name each of the existing files within your "system" folder from:
current45.efdb, fwdb.db3 and despatches.db3
... to e.g."OLD_current45.efdb", etc.. or "CORRUPTED_current45.efdb", etc. - Go to the "backups" folder which is located within the "system" folder, view the files in list mode and sort them by "Date Modified", so you can find the latest backup files.
N.B. If you do not use dispatches within your database, the Date Modified timestamp on the depatches.db3 backup may not be the same as the backups for the current45.efdb and fwdb.db3 files. In this scenario, the three files you are taking from the backups folder should have the same id... name (please see below).
- Copy all 3 files as per info from step "1." with the same "id"-info and paste them into the "system" folder. Now rename each of them according to their file type extension:
current45.efdb, fwdb.db3 and despatches.db3.
Rename example:
id201707271315290000001488.efdb => current45.efdb
id201707271315290000001488.db3 => fwdb.db3
id201707271315290000001488_despatches.db3 => despatches.db3 - Now you are ready to start the fw Server with the recovered backup files.
Failed DB Saves:
As already mentioned above on "How it works:", the "DB Save" process is allowed to fail once!
This can randomly happen on any of the supported OS: Linux, macOS or Windows.
And this can be caused by other running processes (background process of other backups, system protection processes, etc.). Or they can be an early warning sign of underlying disk issues.
And they can also randomly happen, with no explanation to be found on any other log file of the host system.
A failed DB Save process will leave behind corrupted and not usable files, which start with "tmp_db_id" and are located within the fw Server app's "system" folder.
These files are intentionally not cleaned up by the fw Server app on the next start-up, to allow for following troubleshooting.
These need to be manually cleaned up as they can fill up the used disk space.
We recommend checking the fw Server's "system" folder on regular intervals (e.g. at least once per month).