You are not logged in.
also should i be concerned about the errors:
Error description: can't get blocksize, bad .blocklog format?, details: No such file or directoryJust wanted to check as I didnt see it explicitly in the documentation, but do --info and --prune operations only work on a local repo? I get errors when using a remote location for either operation but on the receiver end a local path works. The remote repo path is the exact same as what I use for the working --backup operation.
[root@supernaut:~] /vmfs/volumes/SSD2-Super/XSIBackup-DC/xsibackup --prune root@192.168.246.77:22:/media/wesley/20200626200211 --ssh-ciphers=aes128-ctr --verbosity=10
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.2: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
Primary TMP folder: /tmp/xsi/2171690
-----------------------------------------------------------------------------------------------------------
Secondary TMP folder: /vmfs/volumes/SSD2-Super/XSIBackup-DC/tmp/2171690
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2171690, Running job as: root
-----------------------------------------------------------------------------------------------------------
REGEX check failed for root@192.168.246.77:22:/media/wesley/20200626200211 with pattern ^[-0-9a-zA-Z./_()# ]*$
2020-06-27T18:40:33 | Error code 2090 at file xsibackup.c, line 2090
Error description: when the action is --prune, third argument must be a local or remote path string, passed: root@192.168.246.77:22:/media/wesley/20200626200211
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------[root@supernaut:~] /vmfs/volumes/SSD2-Super/XSIBackup-DC/xsibackup --info root@192.168.246.77:22:/media/wesley --ssh-ciphers=aes128-ctr --verbosity=10
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.2: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
Primary TMP folder: /tmp/xsi/2171716
-----------------------------------------------------------------------------------------------------------
Secondary TMP folder: /vmfs/volumes/SSD2-Super/XSIBackup-DC/tmp/2171716
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2171716, Running job as: root
-----------------------------------------------------------------------------------------------------------
REGEX check failed for root@192.168.246.77:22:/media/wesley with pattern ^[-0-9a-zA-Z./_()# ]*$
2020-06-27T18:40:42 | Error code 2036 at file xsibackup.c, line 2036
Error description: you must set a valid repo path as the second argument after --info
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------on remote:
[root@supernaut:~] /vmfs/volumes/SSD2-Super/XSIBackup-DC/xsibackup --info root@192.168.246.77:22:/media/wesley --ssh-ciphers=aes128-ctr --verbosity=1
0
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.2: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
Primary TMP folder: /tmp/xsi/2171716
-----------------------------------------------------------------------------------------------------------
Secondary TMP folder: /vmfs/volumes/SSD2-Super/XSIBackup-DC/tmp/2171716
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2171716, Running job as: root
-----------------------------------------------------------------------------------------------------------
REGEX check failed for root@192.168.246.77:22:/media/wesley with pattern ^[-0-9a-zA-Z./_()# ]*$
2020-06-27T18:40:42 | Error code 2036 at file xsibackup.c, line 2036
Error description: you must set a valid repo path as the second argument after --info
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------found this post which i think can work!
but would be cool if it were an option. thank you!
[url]https://33hops.com/forum/viewtopic.php?pid=2001#p2001[/url]
I would love to build on the RUNNING / ALL VM identification feature of DC. There are some occurances where I desire to use:
--backup "VMs(ALL)"but would like to exclude 1 or 2 VMs by name. The reason I am using ALL is because DC is running within a cluster and I cant guarantee that vCenter has not migrated a VM from one host to another, so cannot have a static list. But I do not want to backup a couple VMs if they are present.
Would this be possible for you to consider?
The only other thing I may be able to do now is use the
--exclude [=REGEXP]to match the actual files.
Thank you!
have you tried the --replica option? this would work as you desire and it should skip over similar portions of the file on both ends. I would have you try that over ssh and see how it goes.
Testing out the appliance - really cool how you can browse the backups!
In my first test it states that once you close the restore window in the menu that it will stop samba. Samba kept running for me.
Not exactly - for example if i do that on my repo right now i get:
[root@localhost:~] ls -la /vmfs/volumes/Offsite/
total 1720
drwxrwxrwx 46 root root 4096 Jun 24 01:39 .
drwxr-xr-x 1 root root 512 Jun 24 14:55 ..
drwx------ 3 root root 4096 Jun 19 16:57 .vSphere-HA
-rw-r--r-- 1 root root 51 Jun 17 11:45 .xsitools
drwxr-xr-x 3 root root 4096 Jun 17 11:45 20200617114516
drwxr-xr-x 3 root root 4096 Jun 17 11:46 20200617114624
drwxr-xr-x 3 root root 4096 Jun 17 12:51 20200617125109
drwxr-xr-x 4 root root 4096 Jun 17 14:28 20200617142748
drwxr-xr-x 4 root root 4096 Jun 17 16:20 20200617162023
drwxr-xr-x 4 root root 4096 Jun 17 16:56 20200617165554
drwxr-xr-x 3 root root 4096 Jun 17 17:22 20200617172243
drwxr-xr-x 4 root root 4096 Jun 17 17:23 20200617172309
drwxr-xr-x 3 root root 4096 Jun 17 17:39 20200617173949
drwxr-xr-x 4 root root 4096 Jun 17 17:41 20200617174111
drwxr-xr-x 3 root root 4096 Jun 17 18:21 20200617182141
drwxr-xr-x 2 root root 4096 Jun 17 21:02 20200617210206
drwxr-xr-x 3 root root 4096 Jun 17 21:02 20200617210237
drwxr-xr-x 3 root root 4096 Jun 17 21:55 20200617215509
drwxr-xr-x 3 root root 4096 Jun 18 02:24 20200618022308
drwx------ 3 root root 4096 Jun 18 11:44 20200618114358
drwx------ 3 root root 4096 Jun 18 13:24 20200618132435
drwxr-xr-x 3 root root 4096 Jun 18 13:25 20200618132527
drwx------ 3 root root 4096 Jun 18 18:57 20200618185725
drwx------ 7 root root 4096 Jun 19 04:40 20200618191033
drwx------ 5 root root 4096 Jun 19 14:14 20200619104231
drwx------ 3 root root 4096 Jun 19 10:43 20200619104313
drwx------ 3 root root 4096 Jun 19 14:19 20200619141951
drwx------ 3 root root 4096 Jun 19 14:20 20200619142021
drwx------ 3 root root 4096 Jun 19 14:46 20200619144550
drwx------ 3 root root 4096 Jun 19 15:09 20200619150900
drwx------ 3 root root 4096 Jun 19 15:54 20200619155451
drwx------ 3 root root 4096 Jun 19 16:17 20200619161732
drwx------ 4 root root 4096 Jun 19 16:28 20200619162800
drwx------ 3 root root 4096 Jun 19 16:44 20200619164401
drwx------ 3 root root 4096 Jun 19 16:46 20200619164623
drwx------ 3 root root 4096 Jun 19 17:15 20200619171512
drwx------ 3 root root 4096 Jun 20 11:45 20200620114512
drwx------ 4 root root 4096 Jun 24 01:29 20200624012917
drwx------ 4 root root 4096 Jun 24 01:31 20200624013145
drwx------ 4 root root 4096 Jun 24 01:33 20200624013310
drwx------ 8 root root 4096 Jun 24 14:24 20200624013504
drwx------ 3 root root 4096 Jun 24 01:36 20200624013649
drwx------ 3 root root 4096 Jun 24 01:37 20200624013738
drwx------ 5 root root 4096 Jun 24 02:37 20200624013857
drwxr-xr-x 2 root root 4096 Jun 17 11:36 @Recently-Snapshot
drwx------ 3 root root 4096 Jun 19 17:05 Test
drwxr-xr-x 18 root root 4096 Jun 24 14:55 data
-rwx--x--x 1 root root 1382514 Jun 17 11:44 xsibackup
-rw------- 1 root root 2590 Jun 17 11:45 xsibackup_id_rsa
-rw-r--r-- 1 root root 565 Jun 17 11:45 xsibackup_id_rsa.pubwhich doesnt really identify VMs at all - i could do
ls -la /some/repository/*which shows to the vm level but is hard to read. Ideally a usecase would be:
- I need to restore a vm
- query the backup repo for all possible revisions of that VM within a repository
- issue restore command for certain vm revision
depending on the backup schedule I cant assume that all of those time stamped folders contain all vms. make sense?
thanks!
This looks like a great feature!
[quote]Once you have access to (c)XSIBackup-DC repositories via NFS 3/4 mounts, the menu allows to mount the
contents of any -flat.vmdk disk in /mnt/restore/guest, browse files and send them by e-mail from the
built in e-mail client or you can browse the contents of the mounted -flat.vmdk disk via Samba/CIFS
from any Windows explorer.[/quote]
Just a feature request. Would be nice to have elapsed time listed on a restore operation just like backup/replica. Thanks!
Thank you - will take a look! Good to hear either way I am on the right path with a reliable fuse. Thanks.
Yes i understand the --restore option which needs to be pointed to a vm within a timestamped folder, but was imagining a way to list the number of restore points given a certain VM for instance. example:
./xsibackup --list "VMs(YOURVM)" /some/repository
20200622030451
20200623033451
20200624032001or something like that. but then again maybe thats the job of a find command.
As i am starting to use backup repos i just find many time stamped folders and any number of VMs can be under each.
Just kicking around some ideas and wondering what others may be doing if they want another copy of the backups in some sort of cloud destination (ie. s3). some other backup programs allow for direct connection to an s3 based back end, but in my understanding of how xsibackup dc works this would not be possible without a binary on the receiver. If someone wanted to backup to a cloud destination, is the best option just to backup to a local/remote repository on a controlled linux instance or NAS and then use something different to mirror the backup repository to a cloud destination?
as crazy as it may be i wonder if a lightweight linux box could utilize s3fuse and have xsibackup create a repository there.
just wanted to see what others may be doing if anything. thanks!
I would also echo the desire for a simple way to point xsibackup to a backup repo and output what vm's are available for restore.
Hi there.
I am a new DC user and still trying to get my head around a few things.
I have successfully started using it to backup VMs to both ssh and NFS destinations. My question is in regards to the behavior of repeated --backup calls in regards to deduplication/block checks and speed.
As expected the first backup takes some time with the increase in speed mainly coming from deduplication/sparseness. Subsequent --backup calls on the same vm however seem to take longer than I would expect and I want to know if this behavior is normal within reason or if I have done something wrong.
Each time I call --backup to my destination repository it states:
New backup/replica:Does this imply that it is treating this as a brand new backup which will not be utilizing previously uploaded blocks via .map files etc?
The comparison of blocks seems slow and I see data flowing from the server which implies that comparison/block check is not happening locally.
Increasing verbosity I do see each line showing something like:
Block 66, exists: 1, sha1str: 637376502371b9d41846eb086b964be061548207
::: detail ::: 0.11% done | block 67 out of 61440 | Done 0.09%exists: 1means its a match to a previously stored block I would presume?
From a second backup run from a vm that was previously backedup, it took nearly 2 hours to compare and complete while the server is not changing that much. The server has ~ 1 TB allocated to it and about ~ 470 GB of data utilization according to vCenter. I dont think this is expected behavior?
I just ran a quick test on a vm that was never backed up before. please see below the first and then second run. The second run was almost as long as the first. Any pointers on what to do/look for would be very helpful. Thank you!
FIRST RUN:
./xsibackup --backup "VMs(vSphere_Replication)" /vmfs/volumes/Offsite
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: XXX
-----------------------------------------------------------------------------------------------------------
PID: 6912971, Running job as: root
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 10485760 bytes (10.00 MB)
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /vmfs/volumes/Offsite
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: vSphere_Replication
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : vSphere_Replication (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: vSphere_Replication | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-19 14:45:59
-----------------------------------------------------------------------------------------------------------
2020-06-19 14:45:59 | Backing up 27 files, total size is 36746453608
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/27 vSphere_Replication-62816573.hlog 92.00 B | Done 0.00%
-----------------------------------------------------------------------------------------------------------
2/27 vSphere_Replication.vmx 3.11 KB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
3/27 vSphere_Replication-flat.vmdk 9.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 922 out of 922 | Done 26.30%
-----------------------------------------------------------------------------------------------------------
4/27 vSphere_Replication.vmdk 538.00 B | Done 26.30%
-----------------------------------------------------------------------------------------------------------
5/27 vSphere_Replication_1-flat.vmdk 17.00 GB | Done 26.30%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 1741 out of 1741 | Done 75.98%
-----------------------------------------------------------------------------------------------------------
6/27 vSphere_Replication_1.vmdk 541.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
7/27 vSphere_Replication.vmsd 553.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
8/27 vmx-vSphere_Replication-2238554927-1.vswp [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
9/27 vSphere_Replication.vmx.lck [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
10/27 hbr-persistent-state-RDID-5c50bca2-cd60-4837-8f60-c6f2 [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
11/27 vmware-1.log 244.99 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
12/27 vSphere_Replication.vmx~ 3.09 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
13/27 vSphere_Replication.nvram 8.48 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
14/27 vmware-2.log 193.74 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
15/27 vmware-3.log 201.37 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
16/27 hbr-persistent-state-RDID-d7a5248e-f2ca-4a35-ad2f-c86d [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
17/27 vmware-4.log 234.00 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
18/27 vmware-5.log 223.60 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
19/27 vmware.log 213.05 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
20/27 vSphere_Replication-856da32f.vswp [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
21/27 vSphere_Replication.vmsd.tmp 0.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
22/27 vSphere_Replication.vmx.tmp 3.09 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
23/27 vSphere_Replication-Snapshot1.vmsn 19.72 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
24/27 vSphere_Replication-000001-sesparse.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
25/27 vSphere_Replication-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
26/27 vSphere_Replication_1-000001-sesparse.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
27/27 vSphere_Replication_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
Total size: 26625 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-19 15:08:10
-----------------------------------------------------------------------------------------------------------
Time taken: 1331 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 1331 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 20.00 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 60.43 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------SECOND RUN:
./xsibackup --backup "VMs(vSphere_Replication)" /vmfs/volumes/Offsite
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: XXX
-----------------------------------------------------------------------------------------------------------
PID: 6913851, Running job as: root
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 10485760 bytes (10.00 MB)
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /vmfs/volumes/Offsite
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: vSphere_Replication
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : vSphere_Replication (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: vSphere_Replication | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-19 15:09:17
-----------------------------------------------------------------------------------------------------------
2020-06-19 15:09:17 | Backing up 27 files, total size is 36746546791
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/27 vSphere_Replication-62816573.hlog 92.00 B | Done 0.00%
-----------------------------------------------------------------------------------------------------------
2/27 vSphere_Replication.vmx 3.11 KB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
3/27 vSphere_Replication-flat.vmdk 9.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 922 out of 922 | Done 26.30%
-----------------------------------------------------------------------------------------------------------
4/27 vSphere_Replication.vmdk 538.00 B | Done 26.30%
-----------------------------------------------------------------------------------------------------------
5/27 vSphere_Replication_1-flat.vmdk 17.00 GB | Done 26.30%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 1741 out of 1741 | Done 75.98%
-----------------------------------------------------------------------------------------------------------
6/27 vSphere_Replication_1.vmdk 541.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
7/27 vSphere_Replication.vmsd 553.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
8/27 vmx-vSphere_Replication-2238554927-1.vswp [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
9/27 vSphere_Replication.vmx.lck [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
10/27 hbr-persistent-state-RDID-5c50bca2-cd60-4837-8f60-c6f2 [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
11/27 vmware-1.log 244.99 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
12/27 vSphere_Replication.vmx~ 3.09 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
13/27 vSphere_Replication.nvram 8.48 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
14/27 vmware-2.log 193.74 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
15/27 vmware-3.log 201.37 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
16/27 hbr-persistent-state-RDID-d7a5248e-f2ca-4a35-ad2f-c86d [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
17/27 vmware-4.log 234.00 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
18/27 vmware-5.log 223.60 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
19/27 vmware.log 257.01 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
20/27 vSphere_Replication-856da32f.vswp [open excluded]| Done 75.98%
-----------------------------------------------------------------------------------------------------------
21/27 vSphere_Replication.vmsd.tmp 43.00 B | Done 75.98%
-----------------------------------------------------------------------------------------------------------
22/27 vSphere_Replication.vmx.tmp 3.09 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
23/27 vSphere_Replication-Snapshot2.vmsn 19.72 KB | Done 75.98%
-----------------------------------------------------------------------------------------------------------
24/27 vSphere_Replication-000001-sesparse.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
25/27 vSphere_Replication-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
26/27 vSphere_Replication_1-000001-sesparse.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
27/27 vSphere_Replication_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
Total size: 26625 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-19 15:26:30
-----------------------------------------------------------------------------------------------------------
Time taken: 1033 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 1033 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 25.77 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 73.35 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------Thank you. Since I have not purchased yet at this point what would be the best point of contact for support? I see a contact form on the website is that what I should use? Thank you.
Sadly ls gave the same result. Also judging on data speed and time taken I believe it transferred the whole amount:
60G -rw-r--r-- 1 admin root 60G Jun 15 22:21 Radius-flat.vmdkMore troubling and maybe something indicating a different issue my setup has that is causing this, the replica vmdk seems to be unusable. I brought that replica online in the remote esxi environment and the disk was indicated to be unformatted. I am unsure if this would really work, but i put the vmdk-flat file through less it seems blank.
When I do the same command but --backup instead of --replica, the backup goes very fast, and creates an extremely small repository. I was able to get the same result over ssh or local NFS mount. Doing a restore of the backup creates a blank 60 GB vmdk as well.
I have included the output of various commands below to help in figuring out what is going wrong? These commands were directed at a much smaller VM for quicker testing - thin provisioned and had the same result. The host is running ESXi 6.7.0 (Build 8169922).
/vmfs/volumes/SSD/XSIBackup-DC/xsibackup --replica "VMs(Auvik)" root@X.X.X:22:/volume1/vertical --ssh-ciphers=aes128-ctrreplica:
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
SSH: Mon Jun 15 21:33:06 EDT 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2100191, Running job as: root
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC replicating data to /volume1/vertical
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: Auvik
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : Auvik (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: Auvik | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-16 01:44:28
-----------------------------------------------------------------------------------------------------------
2020-06-16 01:44:28 | Backing up 24 files, total size is 10872008991
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/24 Auvik-flat.vmdk 8.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
2/24 Auvik.vmdk 565.00 B | Done 79.01%
-----------------------------------------------------------------------------------------------------------
3/24 Auvik_1-flat.vmdk 1024.00 KB | Done 79.01%
-----------------------------------------------------------------------------------------------------------
4/24 Auvik_1.vmdk 560.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
5/24 Auvik.nvram 8.48 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
6/24 vmware-8.log 201.22 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
7/24 vmware-9.log 196.03 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
8/24 Auvik.vmsd.tmp 43.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
9/24 Auvik.vmx.tmp 2.53 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
10/24 vmware-10.log 193.21 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
11/24 vmware-11.log 116.99 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
12/24 vmware-12.log 223.09 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
13/24 vmx-Auvik-3446253671-1.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
14/24 Auvik.vmx.lck [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
15/24 vmware-13.log 168.58 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
16/24 vmware.log 230.47 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
17/24 Auvik-cd69a867.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
18/24 Auvik-Snapshot4.vmsn 19.16 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
19/24 Auvik-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
20/24 Auvik-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
21/24 Auvik_1-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
22/24 Auvik_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
23/24 Auvik.vmx 2.54 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
24/24 Auvik.vmsd 501.00 B | Done 79.03%
-----------------------------------------------------------------------------------------------------------
Total size: 8194 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-16 02:03:50
-----------------------------------------------------------------------------------------------------------
Time taken: 1162 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 1162 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 7.05 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 7.92 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
SSH session was closed OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------source:
[root@esxi2:/vmfs/volumes/662879e1-2d0c7453-0000-000000000000/Auvik] du -ksh Auvik-flat.vmdk
4.1G Auvik-flat.vmdk
[root@esxi2:/vmfs/volumes/662879e1-2d0c7453-0000-000000000000/Auvik] ls -lash Auvik-flat.vmdk
4290308 -rw------- 1 root root 8.0G Jun 16 02:33 Auvik-flat.vmdkdest:
sh-4.3# du -ksh Auvik-flat.vmdk
8.0G Auvik-flat.vmdk
sh-4.3# ls -lash Auvik-flat.vmdk
8.0G -rw-r--r-- 1 root root 8.0G Jun 15 21:56 Auvik-flat.vmdkbackup:
[root@esxi2:~] /vmfs/volumes/SSD/XSIBackup-DC/xsibackup --backup "VMs(Auvik)" root@X.X.X:22:/volume1/vertical/xsi --ssh-ciphers=aes128-ctr
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
SSH: Mon Jun 15 21:53:01 EDT 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2100963, Running job as: root
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 1048576 bytes (1024.00 KB)
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /volume1/vertical/xsi
-----------------------------------------------------------------------------------------------------------
.xsitools file created at: root@X.X.X.X:22:/volume1/vertical/xsi/.xsitools
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: Auvik
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : Auvik (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: Auvik | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-16 02:04:27
-----------------------------------------------------------------------------------------------------------
2020-06-16 02:04:27 | Backing up 24 files, total size is 10872042917
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/24 Auvik-flat.vmdk 8.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 8192 out of 8192 | Done 79.01%
-----------------------------------------------------------------------------------------------------------
2/24 Auvik.vmdk 565.00 B | Done 79.01%
-----------------------------------------------------------------------------------------------------------
3/24 Auvik_1-flat.vmdk 1024.00 KB | Done 79.01%
-----------------------------------------------------------------------------------------------------------
4/24 Auvik_1.vmdk 560.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
5/24 Auvik.nvram 8.48 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
6/24 vmware-8.log 201.22 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
7/24 vmware-9.log 196.03 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
8/24 Auvik.vmsd.tmp 43.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
9/24 Auvik.vmx.tmp 2.53 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
10/24 vmware-10.log 193.21 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
11/24 vmware-11.log 116.99 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
12/24 vmware-12.log 223.09 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
13/24 vmx-Auvik-3446253671-1.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
14/24 Auvik.vmx.lck [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
15/24 vmware-13.log 168.58 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
16/24 vmware.log 263.37 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
17/24 Auvik-cd69a867.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
18/24 Auvik-Snapshot5.vmsn 19.16 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
19/24 Auvik-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
20/24 Auvik-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
21/24 Auvik_1-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
22/24 Auvik_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
23/24 Auvik.vmx 2.54 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
24/24 Auvik.vmsd 500.00 B | Done 79.03%
-----------------------------------------------------------------------------------------------------------
Total size: 8194 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-16 02:04:46
-----------------------------------------------------------------------------------------------------------
Time taken: 19 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 19 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 431.28 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 2.36 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
SSH session was closed OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------destination repo info:
sh-4.3# xsibackup --info /volume1/vertical/xsi
Missing SSL key pair, generating...
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: Linux, Kernel 4 Major 4 Minor 59 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
PID: 30509, Running job as: root
-----------------------------------------------------------------------------------------------------------
Deduplicated repository information
-----------------------------------------------------------------------------------------------------------
Compression: 1
Block size: 1048576
Block count: 15
-----------------------------------------------------------------------------------------------------------
Retrieving block information...
-----------------------------------------------------------------------------------------------------------
| Real data | Hosted data |
-----------------------------------------------------------------------------------------------------------
Block number | 15 | 8208 |
Data volume | 362.69 KB | 8.00 GB |
Compression | 100.00% | - |
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------Thanks for the help - I really would love to get this to work.
Good morning.
In my first test of DC to replicate a VM over IP, it seems to be creating a thick version of the vm at the remote site. Looking at the docs it seems that this should not be the case and wonder where I made a mistake.
the replica command being used:
./xsibackup --replica "VMs(Radius)" root@X.X.X.X:22:/volume1/backup --ssh-ciphers=aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbcso far my progress is:
Backup start date: 2020-06-14 13:42:37
-----------------------------------------------------------------------------------------------------------
2020-06-14 13:42:37 | Backing up 15 files, total size is 64425634867
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/15 Radius-flat.vmdk 60.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 55.37% done | block 34022 out of 61440 | Done 55.37%looking at the source flat file specifically with du:
15.4G Radius-flat.vmdklooking st the current destination flat file:
34G Radius-flat.vmdk.tmpWhat am i missing here? Thank you.
Thank you for the direction. Thanks for pointing out the config file under ./etc as that looks very useful as well.
I was able to get it to start by tweaking the ssh ciphers as stated. Interestingly enough, the synology is not the issue in my case but the ESXi host. When running
ssh -Q cipheron the ESXi host, it is missing
blowfish-cbc
cast128-cbcas well as:
chacha20-poly1305@openssh.comThese ciphers are all present on the Synology. I found another thread on the forum stating that you found that chacha20-poly1305@openssh.com is unavailable in ESXi 6.7 which is what my test host is running. Either way, defining a compatible set it working now, and this is over a WAN connection so we should be good. Thank you.
Thanks. I am testing between ESXi 6.7 and a remote synology over ssh. The synology is set with ssh/keys etc and i am able to auth as root. The xsibackup binary is in /usr/bin on the synology with execute permissions. Destination directory should be good as we are logging in as root and permissions are good.
The only things in error.log locally or remote are from previous runs with obvious errors.
The command I am running:
./xsibackup --replica "VMs(Radius)" root@X.X.X.X:22:/volume1/backup --compressAt the destination under /volume1/backup it creates the folder for the VM as well as an empty .map folder within. No other files are present.
The output of the command:
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
SSH: Sat Jun 13 17:03:27 EDT 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
PID: 2100055, Running job as: root
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC replicating data to /volume1/vertical
-----------------------------------------------------------------------------------------------------------
Unknown cipher type 'aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc'
Skipping invalid VM '13'
Skipping invalid VM '13'
New backup/replica: Radius | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-13 21:14:44
-----------------------------------------------------------------------------------------------------------
2020-06-13 21:14:44 | Backing up 15 files, total size is 64425637988
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/15 Radius-flat.vmdk 60.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------
SIGTERM (13) condition was trapped: check logs for more details
-----------------------------------------------------------------------------------------------------------
Cleaning up...
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------Thank you for your help! Wish the logs actually said something.
I am testing out DC on my pre-ESXi 7.0 hypervisor. I am getting this error:
SIGTERM (13) condition was trapped: check logs for more detailsbut the logs dont show anything. in researching the release history is this because the trial link provides v1.3.0.1? Is there a way to get the compatible 1.1.1.2 version for testing?
Thanks.