©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


Forum menu Menu

You are not logged in.

Pages: 1 2 3 4 5

#51 Feature requests & improvements » XSIBackup-DC 1.5.1.1: VM mismatch --replica=cbt / new backup location » 2021-09-25 20:38:20

herrep
Replies: 1

Hi,

After having enabled CBT for a VM and performed the first full replication, I realized that the backup location was wrong. Therefore, I manually unregistered and deleted the resulting _XSIREP VM and performed --reset-cbt followed by an --enable-cbt for this CBT to start over again at the new backup location. Furthermore, I already run the suggested commands to overcome registration issues:

/etc/init.d/hostd restart
/etc/init.d/vpxa restart

While the CBT replication seems to work fine for other VMs where I did not had this issue with a wrong backup location in the first place, VmWare still has stored some information concerning the ctk enabled disks so that the new set up replication with option --replication=cbt and --options=R failed. The error log shows:

2021-09-25T20:15:04 | Error code 1168 at file esxi.c, line 1168 | Error description: can't remove all snapshots for VM -1
2021-09-25T20:15:04 | Error code 3540 at file xsibackup.c, line 3540 | Error description: can't remove all snapshots for VM <myvm>
2021-09-25T20:15:05 | Error code 3574 at file xsibackup.c, line 3574 | Error description: could not unregister VM(101) <myvm>_XSIREP, error:

However, I found out that --reset-cbt is not sufficient. It is required to manually disable CBT in the VM, i.e. ctkenable to false for both the general settings as well as for the respective disk. I did this with a powered off VM and without snapshots.  Then I could start over again without issues.

Would it be possible to integrate the disabling of ctk also into --reset-cbt?

Best regards,
Peter

#52 General matters » XSIBackup-DC 1.5.1.1: Which files to exclude from backup --replica=cbt » 2021-09-25 20:13:03

herrep
Replies: 9

Hi,

When replicating a VM using option --replica=cbt in a quiesced hot backup, I realize that each time a quiesce manifest .xml file is created and a snapshot .vmsn file. Besides the vmware-*.log files. So the files get more and more.

Is it fine to delete these files in regular intervals from the VM directory?
Is it fine to exclude these files from replication?

Best regards,
Peter

#53 General matters » XSIBackup-DC 1.5.1.1: CBT count increased by 4 each replication » 2021-09-25 20:02:33

herrep
Replies: 1

Hi,

When I created the first CBT based replication for a number of VMs, the log file showed CBT 1 for each of these VMs when executing the replication with option --replica=cbt.

On subsequent runs, I would expect that CBT is always incremented by 1 so that we have CBT 2, then CBT 3 and so on.

However, what I realized was that CBT was increased by 4 for each run, so the sequence was CBT 1, CBT 5, CBT 9, CBT 13.

Is there anythig I have to worry because of this?

I made a simple test where I just executed another replication run right after having finished the previous one. So I cannot image that another process was interfering in between that caused an extra increment.

Best regards,
Peter

#54 Feature requests & improvements » XSIBackup-DC 1.5.1.1: Warning message Not enough space to backup VM » 2021-09-25 17:40:11

herrep
Replies: 1

Hi,

When performing a cold backup of my virtual machines, XSIBackup-DC 1.5.1.1 performs a calculation of the required space to perform the backup. It appears that all files are taken into consideration that are located in the selected VM directories, even if particular files are excluded from backup through the use of an --exclude regexp pattern.

In my case, I see the following line in the log file:

/!\ Not enough space to backup <vm-name>: avail 9.71 TB | required 40.66 TB

As already mentioned, the backup is successfully performed in my case, as only an amount of data far less than the available space will indeed be backed up. Nevertheless, this warning message is not appropriate. Would it be possible to improve the calculation so that the exclusion pattern will be properly taken into account?

Best regards,
Peter

#55 Re: General matters » XSIBackup-DC --replica=cbt: --options=R appears to be without function » 2021-05-19 12:30:32

herrep

In the mean time, I simplified the backup command as follows:

"/vmfs/volumes/hdd/XSIBackup-DC/xsibackup" \
--replica=cbt \
"VMs(ups-vm)" \
"root@nas:22:/volume1/backup" \
--options=R

Prior to backup, I emptied the vm directory at the target. However, the result was unchanged. The backup as such was reported as successful, but when analyzing the replicated vmx-file, I noticed that it is still unchanged, i.e. no _XSIREP appended. There is also no registration on the ESXi host. Here is the xsibackup.log:

|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.5.0.3: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------
License: 000306E40000000000000000d89d67195a68 | (c)XSIBackup-DC
-------------------------------------------------------------------------------------------------------------
Remote system: Linux
-------------------------------------------------------------------------------------------------------------
PID: 34570689, Running job as: root
-------------------------------------------------------------------------------------------------------------
Remote xsibackup binary found at: /bin/xsibackup
-------------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC replicating data to /volume1/backup
-------------------------------------------------------------------------------------------------------------
Performing --replica action
-------------------------------------------------------------------------------------------------------------
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
ups-vm Hardware Version is: 14
-------------------------------------------------------------------------------------------------------------
All snapshots were removed, as ups-vm is engaged in a CBT job
-------------------------------------------------------------------------------------------------------------
Virtual Machine Name: ups-vm
-------------------------------------------------------------------------------------------------------------
Creating snapshot VM : ups-vm (powered on)
-------------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-------------------------------------------------------------------------------------------------------------
Virtual Machine: ups-vm
-------------------------------------------------------------------------------------------------------------
Backup start date: 2021-05-19T11:34:43
-------------------------------------------------------------------------------------------------------------
2021-05-19 11:34:43 | Backing up 26 files, total size is 11.20 GB
-------------------------------------------------------------------------------------------------------------
    NUMBER	                                                   FILE             SIZE          PROGRESS
-------------------------------------------------------------------------------------------------------------
    1/26                                               ups-vm.vmx          2.86 KB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------
    2/26                                ups-vm-flat.vmdk (CBT 17)         10.00 GB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 10240 out of 10240                                      | Done  89.27%
-------------------------------------------------------------------------------------------------------------
    3/26                                              ups-vm.vmdk        596.00 B     | Done  89.27%
-------------------------------------------------------------------------------------------------------------
    4/26                                              ups-vm.vmsd        455.00 B     | Done  89.27%
-------------------------------------------------------------------------------------------------------------
    5/26                                           ups-vm-aux.xml         13.00 B     | Done  89.27%
-------------------------------------------------------------------------------------------------------------
    6/26	                     vmx-ups-vm-1802602466-2.vswp                    [open excluded]
-------------------------------------------------------------------------------------------------------------
    7/26	                                   ups-vm.vmx.lck                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
    8/26                                                 vmware-617.log        219.98 KB    | Done  89.27%
-------------------------------------------------------------------------------------------------------------
    9/26                                                 vmware-618.log        263.72 KB    | Done  89.27%
-------------------------------------------------------------------------------------------------------------
   10/26                                             ups-vm.nvram          8.48 KB    | Done  89.27%
-------------------------------------------------------------------------------------------------------------
   11/26                                                 vmware-619.log        223.89 KB    | Done  89.27%
-------------------------------------------------------------------------------------------------------------
   12/26                                  ups-vm-flat.vmdk___hash         82.00 B     | Done  89.27%
-------------------------------------------------------------------------------------------------------------
   13/26                                                     vmware.log        333.84 KB    | Done  89.27%
-------------------------------------------------------------------------------------------------------------
   14/26                                              ups-vm.vmx~          2.85 KB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   15/26                                          ups-vm.vmsd.tmp         45.00 B     | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   16/26                                           ups-vm.vmx.tmp          2.85 KB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   17/26                                                 vmware-614.log        220.84 KB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   18/26                                                 vmware-616.log        358.92 KB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   19/26                                                 vmware-615.log        220.83 KB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   20/26	                             ups-vm-6b7187e2.vswp                    [open excluded]
-------------------------------------------------------------------------------------------------------------
   21/26                             vmx-ups-vm-1802602466-1.vswp         80.00 MB    | Done  89.28%
-------------------------------------------------------------------------------------------------------------
   22/26	                                  ups-vm-ctk.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   23/26                                  ups-vm-Snapshot536.vmsn         19.48 KB    | Done  89.98%
-------------------------------------------------------------------------------------------------------------
   24/26	                      ups-vm-000001-sesparse.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   25/26	                               ups-vm-000001.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   26/26	                           ups-vm-000001-ctk.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
Total size:                                                                     10.08 GB    | Done 100.00%
-------------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-------------------------------------------------------------------------------------------------------------
Backup end date: 2021-05-19T11:35:37
-------------------------------------------------------------------------------------------------------------
Time taken: 00:00:54 (54 sec.)
-------------------------------------------------------------------------------------------------------------
Total time:       54 sec.
-------------------------------------------------------------------------------------------------------------
Full file speed:	                                                                   191.15 mb/s
-------------------------------------------------------------------------------------------------------------
Real data speed:	                                                                   104.84 mb/s
-------------------------------------------------------------------------------------------------------------
Item backup completed without errors
-------------------------------------------------------------------------------------------------------------
Final checksum: 110351964 bytes were sent and confirmed to have been written remotely
-------------------------------------------------------------------------------------------------------------
Data processing completed successfully
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Unlocked backup               OK
-------------------------------------------------------------------------------------------------------------

After this test, I run --reset-cbt, deleted the directory of that vm at the target and started again. Besides that now a CBT 1 full cycle was started, I had the same result of no changed vm name and no registration.

As the xsibackup.log is entirely silent about what is going on, is there any option to increase verbosity to see what is indeed happening?

#56 Re: General matters » XSIBackup-DC --replica=cbt: --options=R appears to be without function » 2021-05-18 15:49:19

herrep

I understand that using --exclude might cause issues.  However, I already re-run the command on other VMs where I did not need to exclude anything so that I could omit --exclude.  So the issue does not appear to be related to the --exclude option.

As already reported, I had absolutely no issues with XSIBackup-Pro and --backup-prog=Onediff.  The replica were written, the display name in the replicated vmx file got appended by _XSIBAK and verything worked fine. The only change I applied was activating ctkEnable and using XSIBackup-DC 1.5.0.3.  Everything else remained untouched.

Please let me know whether I can provide any further log files.

#57 General matters » XSIBackup-DC --subject: No status output » 2021-05-18 13:13:17

herrep
Replies: 1

Hi,

Originally, I used XSIBackup-Pro and was  happy about the --description option to add some user-specified text to the subject line of the generated email. So the subject of the email report contained a brief note on the success of the backup plus a note related to the type of backup as specified by --description.

After migrating to XSIBackup-DC, I learned that --description was no longer there, but --subject. Unfortunately, --subject replaces the entire subject of the generated email so that there is no brief note on the success of the backup any longer in the subject.

Is there a way to have a similar functionality in XSIBackup-DC like it was in XSIBackup-Pro, i.e. reporting on errors in the subject line plus some additional user-specified information?

Best regards,
Peter

#58 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 10:43:13

herrep

My previous message appears to have crossed. The issue was that a superflous .vmdk file was in the VM directory. After removing it, the backup worked fine. Thanks a lot for your support!

#59 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 10:37:23

herrep

After checking the issue again, I realized that there was a superflous .vmdk file in the VM's directory which was copied a long time ago. Clearly, this file was not adapted to ctkEnabled as it is not referenced in the .vmx file. I removed the file and started xsibackup again with --replica=cbt. The backup succeeded! Probably it would make sense to catch this error in xsibackup.

#60 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 10:17:11

herrep

I removed --quiesce and switched to --backup-how=cold to avoid any issues resulting from hot backups as there are databases running there. Unfortunately, the result is the same.

2021-05-18T10:13:42 | Error code 194 at file signal.c, line 194 | Error description: raised SIGTERM (11) (2) in job, num of errors: 11, check error.log

#61 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 07:48:55

herrep

I further investigated the issue and found out that --replica=cbt is successfully executed for all other more than 10 VMs. There is only one single VM which causes the error. Are there any other log files I should provide?

#62 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 07:47:53

herrep

Anyway, the problem is related to one out of three VMs. Two other VMs can be replicated by --replica=cbt.  Do you see any hint in the provided vmware.log? Is the snapshot correctly provided by the ESXi host?

#63 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-18 07:45:18

herrep

I have removed the ambigous action --check=full from the command. Furthermore, I have run --reset-cbt prior to perform --replica=cbt again and fully deleted the replication directory at the target. Nevertheless, the output is the same negative result.

It appears that you have properly implemented a priority of the action --replica=cbt over the action --check=full if both actions are present in the same command. I do not see any difference in the output when omitting the superflous action --check=full.

#64 Re: General matters » XSIBackup-DC --backup: Use of --rotate in a deduplicated backup repo » 2021-05-17 10:35:41

herrep

I read your post about pruning and your concerns. I assume these concerns also apply to --rotate?

Would you rather suggest to start a new repo each month instead of using --rotate=30 to keep the latest 30 daily backups? And then deleting an entire outdated monthly repo?

#65 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » 2021-05-17 10:29:47

herrep

Thank you very much. Understood. I already contacted our mail admin.

#66 Re: General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-17 10:28:39

herrep

The --remote-path=/usr/bin/xsibackup is just a reminder for me so that I do not forget to update xsibackup in the NAS upon performing an update of xsibackup on the ESXi host. Or is there a mechanism that prevents one from using different xsibackup versions between client and server?

#67 Re: General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-17 10:23:55

herrep

I have updated the original post to show the full command and also an excerpt of the vmware.log while executing the --replica=cbt action.

#68 Re: General matters » XSIBackup-DC --replica=cbt: --options=R appears to be without function » 2021-05-17 09:54:19

herrep

The original post is now updated so that it also shows the full command in addition to the already provided xsibackup log output.

From the log output, the CBT cycles appear to work, as after a second and now a third run, I can see more or less the same output as for the first run, but instead of "CBT 1 full" I can see "CBT 3".

However, due to the use of --options=R I would expect that the replicated VM is registered under the name vm_XSIREP. But this does not happen although the ESXi host can access the backup directory (NFS mount). It already worked fine with XSIBackup-Pro. I can also not see that the replicated vm.vmx includes a VM name that has been appended by _XSIREP. It is still a copy of the original VM.

#69 General matters » XSIBackup-DC --replica=cbt: Error code 194 at file signal.c, line 194 » 2021-05-16 22:53:51

herrep
Replies: 12

Hi,

While some of my VMs could be replicated by using XSIBackup-DC 1.5.0.3 with --replica=cbt action, there is at least one VM that reports an error when invoking the --replica=cbt action with --options=R.

This is the command I executed right after having enabled cbt for the VMs named vm and vm-failed:

"/vmfs/volumes/hdd1/XSIBackup-DC/xsibackup" \
--replica=cbt \
"VMs(vm,vm-failed)" \
"root@nas:22:/volume1/backup" \
--options=R \
--exclude="data[-.].*vmdk" \
--check=full \
--quiesce \
--config-backup \
--mail-to="<mail-recipient>" \
--use-smtp="1" \
--description="replica hot" \
--backup-how=hot \
--remote-path="/usr/bin/xsibackup"

The first VM named vm was processed correctly, i.e. without reporting any errors. I got a "CBT 1 full" cycle and further cycles are reported as CBT 2, CBT 3. However, there appears an issue with the --options=R even with the successfully created replica of VM named vm. THe log file related to VM vm can be found in [url=https://33hops.com/forum/viewtopic.php?id=949]this forum post[/url].

Here is the xsibackup.log output for the second VM vm-failed:

(c)XSIBackup-DC replicating data to /volume1/nas
-------------------------------------------------------------------------------------------------------------
Performing --replica action
-------------------------------------------------------------------------------------------------------------
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
vm-failed Hardware Version is: 14
-------------------------------------------------------------------------------------------------------------
All snapshots were removed, as vm is engaged in a CBT job
-------------------------------------------------------------------------------------------------------------
Virtual Machine Name: vm-failed
-------------------------------------------------------------------------------------------------------------
Creating snapshot VM : vm-failed (powered on)
-------------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-------------------------------------------------------------------------------------------------------------
Remote target dir is empty, full sync will be performed
-------------------------------------------------------------------------------------------------------------
Virtual Machine: vm-failed
-------------------------------------------------------------------------------------------------------------
Backup start date: 2021-05-16T22:45:10
-------------------------------------------------------------------------------------------------------------
2021-05-16 22:45:10 | Backing up 30 files, total size is 130.33 GB
-------------------------------------------------------------------------------------------------------------
    NUMBER	                                                   FILE             SIZE          PROGRESS
-------------------------------------------------------------------------------------------------------------
    1/30                                                 vm-failed.vmx          3.12 KB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------
ble-cbt="vm-failed" first
-------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------
2021-05-16T22:45:14 | Error code 194 at file signal.c, line 194 | Error description: raised SIGTERM (11) (2) in job, num of errors: 11, check error.log
-------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------
SIGTERM (11) condition was trapped: check logs for more details
-------------------------------------------------------------------------------------------------------------
Cleaning up...
-------------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------

The error log does not show any issue.

From the not fully visible remark above

"ble-cbt="vm-failed" first

I assume that CBT is not enabled.

But this is not correct. I have successfully run --enable-cbt="vm-failed" and a second run --enable-cbt="vm-failed" confirms that cbt is active for vm-failed.

Here is a listing of the files in the vm-failed directory at the ESXi host:

-rw-r--r--    1 root     root            13 May 16 22:45 vm-failed-aux.xml
-rw-r--r--    1 root     root            53 Dec  3 18:46 vm-failed-d6f15467.hlog
-rw-------    1 root     root     2113929216 May 16 22:40 vm-failed-d6f15467.vswp
-rw-------    1 root     root     68718428160 Apr 15  2019 vm-failed-flat.vmdk
-rw-------    1 root     root          8684 May 11 15:48 vm-failed.nvram
-rw-------    1 root     root           609 Apr 15  2019 vm-failed.vmdk
-rw-r--r--    1 root     root            45 May 16 22:45 vm-failed.vmsd
-rw-r--r--    1 root     root            45 May 16 22:45 vm-failed.vmsd.tmp
-rw-r--r--    1 root     root          3192 May 16 22:45 vm-failed.vmx
-rw-------    1 root     root             0 May 16 22:40 vm-failed.vmx.lck
-rw-r--r--    1 root     root          3192 May 16 22:45 vm-failed.vmx.tmp
-rw-------    1 root     root           150 Mar  6 14:40 vm-failed.vmxf
-rw-r--r--    1 root     root          3199 May 16 22:45 vm-failed.vmx~
-rw-------    1 root     root       4194816 May 16 22:45 vm-failed_1-ctk.vmdk
-rw-------    1 root     root     68719476736 May 17 10:12 vm-failed_1-flat.vmdk
-rw-r--r--    1 root     root            82 May 13 04:40 vm-failed_1-flat.vmdk___hash
-rw-------    1 root     root           583 May 16 22:45 vm-failed_1.vmdk

This is the vmware.log output when creating the snapshot while executing the --replica=cbt action for VM vm-failed:

2021-05-16T22:45:05.711Z| vmx| I125: VigorTransportProcessClientPayload: opID=vim-cmd-38-da7e seq=520: Receiving Snapshot.Take request.
2021-05-16T22:45:05.712Z| vmx| I125: SnapshotVMX_TakeSnapshot start: 'xsi40150622346030184', deviceState=0, lazy=0, quiesced=1, forceNative=0, tryNative=1, saveAllocMaps=0
2021-05-16T22:45:05.714Z| vmx| I125: DiskLib_IsVMFSSparseSupported: vmfssparse is not supported on /vmfs/volumes/hdd/vm-failed: f532.
2021-05-16T22:45:05.714Z| vmx| I125: DISKLIB-LIB_CREATE   : DiskLibCreateCreateParam: Selecting the default child type as SeSparse for /vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk.
2021-05-16T22:45:05.714Z| vmx| I125: DISKLIB-LIB_CREATE   : DiskLibCreateCreateParam: seSparse grain size is set to 8 for '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk'
2021-05-16T22:45:05.714Z| vmx| I125: SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (The operation completed successfully).
2021-05-16T22:45:05.726Z| vcpu-0| I125: ToolsBackup: changing quiesce state: IDLE -> STARTED
2021-05-16T22:45:07.729Z| vmx| I125: ToolsBackup: changing quiesce state: STARTED -> COMMITTED
2021-05-16T22:45:07.730Z| vcpu-0| I125: Destroying virtual dev for scsi0:0 vscsi=16167
2021-05-16T22:45:07.730Z| vcpu-0| I125: VMMon_VSCSIStopVports: No such target on adapter
2021-05-16T22:45:07.731Z| vcpu-0| I125: SnapshotVMXTakeSnapshotWork: Transition to mode 0.
2021-05-16T22:45:07.733Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (5) size = 68719476736, hd = 0. Type 3
2021-05-16T22:45:07.733Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:07.733Z| vcpu-0| I125: Closing all the disks of the VM.
2021-05-16T22:45:07.733Z| vcpu-0| I125: Closing disk 'scsi0:0'
2021-05-16T22:45:07.734Z| vcpu-0| I125: DISKLIB-CBT   : Shutting down change tracking for untracked fid 271304579.
2021-05-16T22:45:07.734Z| vcpu-0| I125: DISKLIB-CBT   : Successfully disconnected CBT node.
2021-05-16T22:45:07.740Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:07.741Z| vcpu-0| I125: SNAPSHOT: SnapshotConfigInfoOpenVmsd: Creating new snapshot dictionary, '/vmfs/volumes/hdd/vm-failed/vm-failed.vmsd.usd'.
2021-05-16T22:45:07.867Z| vcpu-0| I125: SNAPSHOT: SnapshotDumperOpenFromInfo: Creating checkpoint file /vmfs/volumes/hdd/vm-failed/vm-failed-Snapshot616.vmsn
2021-05-16T22:45:07.869Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (13) size = 68719476736, hd = 0. Type 3
2021-05-16T22:45:07.870Z| vcpu-0| I125: DiskLib_IsVMFSSparseSupported: vmfssparse is not supported on /vmfs/volumes/hdd/vm-failed: f532.
2021-05-16T22:45:07.870Z| vcpu-0| I125: DISKLIB-LIB_CREATE   : DiskLibCreateCreateParam: Selecting the default child type as SeSparse for /vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk.
2021-05-16T22:45:07.870Z| vcpu-0| I125: DISKLIB-LIB_CREATE   : DiskLibCreateCreateParam: seSparse grain size is set to 8 for '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk'
2021-05-16T22:45:07.870Z| vcpu-0| I125: DISKLIB-LIB_CREATE   : CREATE CHILD: "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk" -- seSparse cowGran=8 allocType=0 policy=''
2021-05-16T22:45:07.870Z| vcpu-0| I125: DISKLIB-LIB_CREATE   : CREATE-CHILD: Creating disk backed by 'file'
2021-05-16T22:45:07.871Z| vcpu-0| I125: Const Header:
2021-05-16T22:45:07.871Z| vcpu-0| I125+  constMagic     = 0xcafebabe
2021-05-16T22:45:07.871Z| vcpu-0| I125+  version        = 2.1
2021-05-16T22:45:07.871Z| vcpu-0| I125+  capacity       = 134217728
2021-05-16T22:45:07.871Z| vcpu-0| I125+  grainSize      = 8
2021-05-16T22:45:07.871Z| vcpu-0| I125+  grainTableSize = 64
2021-05-16T22:45:07.871Z| vcpu-0| I125+  flags          = 0
2021-05-16T22:45:07.872Z| vcpu-0| I125: Extents:
2021-05-16T22:45:07.872Z| vcpu-0| I125+  Header         : <1 : 1>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  JournalHdr     : <2 : 2>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  Journal        : <2048 : 2048>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  GrainDirectory : <4096 : 2048>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  GrainTables    : <6144 : 262144>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  FreeBitmap     : <268288 : 4096>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  BackMap        : <272384 : 262144>
2021-05-16T22:45:07.872Z| vcpu-0| I125+  Grain          : <534528 : 134217728>
2021-05-16T22:45:07.872Z| vcpu-0| I125: Volatile Header:
2021-05-16T22:45:07.872Z| vcpu-0| I125+ volatileMagic     = 0xcafecafe
2021-05-16T22:45:07.872Z| vcpu-0| I125+ FreeGTNumber      = 0
2021-05-16T22:45:07.872Z| vcpu-0| I125+ nextTxnSeqNumber  = 0
2021-05-16T22:45:07.872Z| vcpu-0| I125+ replayJournal     = 0
2021-05-16T22:45:07.873Z| vcpu-0| I125: DISKLIB-SESPARSE : seSparseExtentCreate: "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : success
2021-05-16T22:45:07.874Z| vcpu-0| I125: DISKLIB-DSCPTR: "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk" : creation successful.
2021-05-16T22:45:07.879Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (1) size = 273678336, hd = 0. Type 19
2021-05-16T22:45:07.915Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:07.915Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:07.915Z| vcpu-0| A100: ConfigDB: Setting scsi0:0.fileName = "vm-failed_1-000001.vmdk"
2021-05-16T22:45:07.919Z| vcpu-0| I125: Checkpoint_Unstun: vm stopped for 190331 us
2021-05-16T22:45:07.919Z| vcpu-0| I125: SCSI: switching scsi0 to push completion mode
2021-05-16T22:45:07.920Z| vcpu-0| A100: ConfigDB: Setting scsi0:0.redo = ""
2021-05-16T22:45:07.920Z| vcpu-0| I125: DISK: OPEN scsi0:0 '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' persistent R[]
2021-05-16T22:45:07.921Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (10) size = 273678336, hd = 279627655. Type 19
2021-05-16T22:45:07.921Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: "vm-failed_1-000001-sesparse.vmdk" (0xa)
2021-05-16T22:45:07.921Z| vcpu-0| I125: DISKLIB-LINK  : Opened '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' (0xa): seSparse, 134217728 sectors / 64 GB.
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (14) size = 68719476736, hd = 309381000. Type 3
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: "vm-failed_1-flat.vmdk" (0xe)
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-LINK  : Opened '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' (0xe): vmfs, 134217728 sectors / 64 GB.
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-CHAINESX : ChainESXOpenSubChain:(0) fid = 309381000, extentType = 2
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-CHAINESX : ChainESXOpenSubChain:(1) fid = 279627655, extentType = 1
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-LIB_BLOCKTRACK   : Resuming change tracking.
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 281659273.
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-CBT   : Successfuly created cbt node 10c9c789-cbt.
2021-05-16T22:45:07.922Z| vcpu-0| I125: DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/10c9c789-cbt
2021-05-16T22:45:07.923Z| vcpu-0| I125: DISKLIB-LIB   : Opened "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk" (flags 0xa, type vmfs).
2021-05-16T22:45:07.923Z| vcpu-0| I125: DISK: Disk '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' has UUID '60 00 c2 9e 1f 21 a6 ed-94 e2 7f 6d 0b 59 44 96'
2021-05-16T22:45:07.923Z| vcpu-0| I125: DISK: OPEN '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' Geo (8354/255/63) BIOS Geo (0/0/0)
2021-05-16T22:45:07.923Z| vcpu-0| I125: Creating virtual dev for 'scsi0:0'.
2021-05-16T22:45:07.923Z| vcpu-0| I125: DumpDiskInfo: scsi0:0 createType=11, capacity = 134217728, numLinks = 2, allocationType = 2
2021-05-16T22:45:07.923Z| vcpu-0| I125: SCSIDiskESXPopulateVDevDesc: Using FS backend
2021-05-16T22:45:07.923Z| vcpu-0| I125: DISKUTIL: scsi0:0 : geometry=8354/255/63
2021-05-16T22:45:07.923Z| vcpu-0| I125: SCSIFilterESXAttachCBRCInt: CBRC not enabled or opened without filters,skipping CBRC           filter attach.
2021-05-16T22:45:07.923Z| vcpu-0| I125: SCSIFilterSBDAttachCBRC: device scsi0:0 is not SBD. Skipping CBRC attach SBD way.
2021-05-16T22:45:07.923Z| vcpu-0| I125: SnapshotVMXTakeSnapshotWork: Transition to mode 1.
2021-05-16T22:45:07.923Z| vcpu-0| I125: ToolsBackup: changing quiesce state: COMMITTED -> FINISHING
2021-05-16T22:45:07.926Z| vcpu-0| I125: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk'
2021-05-16T22:45:07.926Z| vcpu-0| I125: DDB: "longContentID" = "9b885358f9a744640b58f1a5d2a2b710" (was "e551d9491c6ace741e255e6531761462")
2021-05-16T22:45:07.928Z| vcpu-0| I125: DISKLIB-CHAIN : DiskChainUpdateContentID: old=0x31761462, new=0xd2a2b710 (9b885358f9a744640b58f1a5d2a2b710)
2021-05-16T22:45:08.927Z| vmx| I125: ToolsBackup: changing quiesce state: FINISHING -> DONE_WAIT
2021-05-16T22:45:09.929Z| vmx| I125: ToolsBackup: changing quiesce state: DONE_WAIT -> GET_MANIFEST
2021-05-16T22:45:09.991Z| vcpu-0| I125: HGFileCopyCreateSessionCB: Successfully created the session.
2021-05-16T22:45:09.998Z| vcpu-0| I125: ToolsBackup: changing quiesce state: GET_MANIFEST -> IDLE
2021-05-16T22:45:09.998Z| vcpu-0| I125: ToolsBackup: changing quiesce state: IDLE -> DONE
2021-05-16T22:45:10.006Z| vcpu-0| I125: SnapshotVMXTakeSnapshotComplete: Done with snapshot 'xsi40150622346030184': 616
2021-05-16T22:45:10.006Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=vim-cmd-38-da7e seq=520: Completed Snapshot request.
2021-05-16T22:45:14.848Z| vmx| I125: VigorTransportProcessClientPayload: opID=vim-cmd-e2-dac3 seq=704: Receiving Snapshot.Delete request.
2021-05-16T22:45:14.850Z| vmx| I125: SNAPSHOT: SnapshotDeleteWork '/vmfs/volumes/hdd/vm-failed/vm-failed.vmx' : 616
2021-05-16T22:45:14.851Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (5) size = 290459648, hd = 0. Type 19
2021-05-16T22:45:14.851Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:14.851Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (5) size = 68719476736, hd = 0. Type 3
2021-05-16T22:45:14.851Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:14.855Z| vmx| I125: SNAPSHOT: SnapshotDeleteFile deleted '/vmfs/volumes/hdd/vm-failed/vm-failed-quiesce_manifest616.xml'.
2021-05-16T22:45:14.858Z| vmx| I125: VigorTransport_ServerSendResponse opID=vim-cmd-e2-dac3 seq=704: Completed Snapshot request.
2021-05-16T22:45:14.859Z| vmx| I125: VigorTransportProcessClientPayload: opID=vim-cmd-e2-dac3 seq=705: Receiving Snapshot.Consolidate request.
2021-05-16T22:45:14.860Z| vmx| I125: SnapshotVMX_Consolidate: Starting online snapshot consolidate operation.
2021-05-16T22:45:14.860Z| vmx| I125: Vix: [mainDispatch.c:4239]: VMAutomation_ReportPowerOpFinished: statevar=3, newAppState=1880, success=1 additionalError=0
2021-05-16T22:45:14.860Z| vmx| I125: Turning on snapshot info cache. VM=vm-failed.vmx.
2021-05-16T22:45:14.862Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (1114117) size = 0, hd = 0. Type 19
2021-05-16T22:45:14.862Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:14.862Z| vmx| I125: SNAPSHOT: Turning on snapshot disk cache.
2021-05-16T22:45:14.862Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (5) size = 290459648, hd = 0. Type 19
2021-05-16T22:45:14.862Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:14.863Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (5) size = 68719476736, hd = 0. Type 3
2021-05-16T22:45:14.863Z| vmx| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:14.863Z| vmx| I125: WORKER: Creating new group with numThreads=1 (16)
2021-05-16T22:45:14.863Z| vmx| I125: WORKER: Creating new group with numThreads=1 (16)
2021-05-16T22:45:14.863Z| vcpu-0| I125: Destroying virtual dev for scsi0:0 vscsi=16168
2021-05-16T22:45:14.863Z| vcpu-0| I125: VMMon_VSCSIStopVports: No such target on adapter
2021-05-16T22:45:14.863Z| vcpu-0| I125: Closing disk 'scsi0:0'
2021-05-16T22:45:14.865Z| vcpu-0| I125: DISKLIB-CBT   : Shutting down change tracking for untracked fid 281659273.
2021-05-16T22:45:14.865Z| vcpu-0| I125: DISKLIB-CBT   : Successfully disconnected CBT node.
2021-05-16T22:45:14.870Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:14.870Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:14.870Z| vcpu-0| I125: Checkpoint_Unstun: vm stopped for 7494 us
2021-05-16T22:45:14.870Z| vcpu-0| I125: SCSI: switching scsi0 to push completion mode
2021-05-16T22:45:14.871Z| vcpu-0| A100: ConfigDB: Setting scsi0:0.redo = ""
2021-05-16T22:45:14.871Z| vcpu-0| I125: DISK: OPEN scsi0:0 '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' persistent R[]
2021-05-16T22:45:14.872Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (8) size = 290459648, hd = 281855881. Type 19
2021-05-16T22:45:14.872Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: "vm-failed_1-000001-sesparse.vmdk" (0x8)
2021-05-16T22:45:14.872Z| vcpu-0| I125: DISKLIB-LINK  : Opened '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' (0x8): seSparse, 134217728 sectors / 64 GB.
2021-05-16T22:45:14.872Z| vcpu-0| I125: DISKLIB-CHAINESX : ChainESXOpenSubChain:(0) fid = 281855881, extentType = 1
2021-05-16T22:45:14.872Z| vcpu-0| I125: DISKLIB-LIB_BLOCKTRACK   : Resuming change tracking.
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 288081802.
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-CBT   : Successfuly created cbt node 112bc78a-cbt.
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/112bc78a-cbt
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-LIB   : Opened "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk" (flags 0x8, type seSparse).
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (522) size = 68719476736, hd = 266192781. Type 3
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: "vm-failed_1-flat.vmdk" (0x20a)
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-LINK  : Opened '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' (0x20a): vmfs, 134217728 sectors / 64 GB.
2021-05-16T22:45:14.873Z| vcpu-0| I125: DISKLIB-LIB_BLOCKTRACK   : Resuming change tracking.
2021-05-16T22:45:14.874Z| vcpu-0| I125: DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 266192781.
2021-05-16T22:45:14.874Z| vcpu-0| I125: DISKLIB-CBT   : Successfuly created cbt node fddc78d-cbt.
2021-05-16T22:45:14.874Z| vcpu-0| I125: DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/fddc78d-cbt
2021-05-16T22:45:14.874Z| vcpu-0| I125: DISKLIB-LIB   : Opened "/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk" (flags 0x20a, type vmfs).
2021-05-16T22:45:14.875Z| vcpu-0| I125: DISKLIB-CBT   : Shutting down change tracking for untracked fid 288081802.
2021-05-16T22:45:14.875Z| vcpu-0| I125: DISKLIB-CBT   : Successfully disconnected CBT node.
2021-05-16T22:45:14.876Z| vcpu-0| I125: DISKLIB-CBT   : Shutting down change tracking for untracked fid 266192781.
2021-05-16T22:45:14.877Z| vcpu-0| I125: DISKLIB-CBT   : Successfully disconnected CBT node.
2021-05-16T22:45:14.883Z| vcpu-0| I125: DISKLIB-CHAINESX : ChainESXOpenSubChain:(0) fid = 266192781, extentType = 2
2021-05-16T22:45:14.883Z| vcpu-0| I125: DISKLIB-CHAINESX : ChainESXOpenSubChain:(1) fid = 281855881, extentType = 1
2021-05-16T22:45:14.883Z| vcpu-0| I125: DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 288212874.
2021-05-16T22:45:14.884Z| vcpu-0| I125: DISKLIB-CBT   : Successfuly created cbt node 112dc78a-cbt.
2021-05-16T22:45:14.884Z| vcpu-0| I125: DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/112dc78a-cbt
2021-05-16T22:45:14.884Z| vcpu-0| I125: DISK: Disk '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' has UUID '60 00 c2 9e 1f 21 a6 ed-94 e2 7f 6d 0b 59 44 96'
2021-05-16T22:45:14.884Z| vcpu-0| I125: DISK: OPEN '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' Geo (8354/255/63) BIOS Geo (0/0/0)
2021-05-16T22:45:14.884Z| vcpu-0| I125: ConsolidateDiskOpenCB: Opening disk node 'scsi0:0' for combining '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' -> '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk'.
2021-05-16T22:45:14.885Z| vcpu-0| I125: DISKLIB-LIB_MISC   : Opening mirror node /vmfs/devices/svm/112dc78a-fddc78d-svmmirror
2021-05-16T22:45:14.885Z| vcpu-0| I125: Creating virtual dev for 'scsi0:0'.
2021-05-16T22:45:14.885Z| vcpu-0| I125: DumpDiskInfo: scsi0:0 createType=11, capacity = 134217728, numLinks = 2, allocationType = 2
2021-05-16T22:45:14.886Z| vcpu-0| I125: SCSIDiskESXPopulateVDevDesc: Using FS backend
2021-05-16T22:45:14.886Z| vcpu-0| I125: DISKUTIL: scsi0:0 : geometry=8354/255/63
2021-05-16T22:45:14.886Z| vcpu-0| I125: SCSIFilterESXAttachCBRCInt: CBRC not enabled or opened without filters,skipping CBRC           filter attach.
2021-05-16T22:45:14.886Z| vcpu-0| I125: SCSIFilterSBDAttachCBRC: device scsi0:0 is not SBD. Skipping CBRC attach SBD way.
2021-05-16T22:45:15.005Z| worker-34524604| I125: DISKLIB-LIB   : DiskLib_IsCombinePossible: Combine 2 links at 0. Need 0 MB of free space (526382 MB available)
2021-05-16T22:45:15.005Z| vmx| I125: ConsolidateMirrorCombineStart: Combine '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' -> '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' (node = scsi0:0): 2 links starting from 0. usesCurrent = 1, isNative = 0 
2021-05-16T22:45:15.005Z| vmx| I125: Mirror_DiskCopy: Starting disk copy.
2021-05-16T22:45:15.207Z| vcpu-0| I125: Destroying virtual dev for scsi0:0 vscsi=16169
2021-05-16T22:45:15.207Z| vcpu-0| I125: VMMon_VSCSIStopVports: No such target on adapter
2021-05-16T22:45:15.207Z| vcpu-0| I125: Closing disk 'scsi0:0'
2021-05-16T22:45:15.207Z| vcpu-0| I125: ConsolidateDiskCloseCB: Closing disk node 'scsi0:0' after combining '/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001.vmdk' -> '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk'.
2021-05-16T22:45:15.209Z| vcpu-0| I125: DISKLIB-CTK   : Forcing flush of change info for "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-ctk.vmdk".
2021-05-16T22:45:15.215Z| vcpu-0| I125: DISKLIB-CTK   : ChangeTracker_EndCombine()
2021-05-16T22:45:15.218Z| vcpu-0| I125: DISKLIB-CTK   : Unlinked /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk, tmp file: /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk-tmp
2021-05-16T22:45:15.230Z| vcpu-0| I125: DISKLIB-CTK   : resuming /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk-tmp
2021-05-16T22:45:15.231Z| vcpu-0| I125: DISKLIB-CTK   : ChangeTracker_EndCombine: Renaming /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk-tmp to /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk
2021-05-16T22:45:15.231Z| vcpu-0| I125: DISKLIB-CTK   : ChangeTracker_EndCombine: Attempting to unlink /vmfs/volumes/hdd/vm-failed/vm-failed_1-ctk.vmdk-tmp
2021-05-16T22:45:15.232Z| vcpu-0| I125: DDB: "longContentID" = "9b885358f9a744640b58f1a5d2a2b710" (was "e551d9491c6ace741e255e6531761462")
2021-05-16T22:45:15.246Z| vcpu-0| I125: DISKLIB-CBT   : Shutting down change tracking for untracked fid 288212874.
2021-05-16T22:45:15.246Z| vcpu-0| I125: DISKLIB-CBT   : Successfully disconnected CBT node.
2021-05-16T22:45:15.247Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:15.247Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : closed.
2021-05-16T22:45:15.247Z| vcpu-0| A100: ConfigDB: Setting scsi0:0.fileName = "vm-failed_1.vmdk"
2021-05-16T22:45:15.248Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : open successful (1115137) size = 0, hd = 0. Type 19
2021-05-16T22:45:15.248Z| vcpu-0| I125: DISKLIB-LIB_BLOCKTRACK   : Resuming change tracking.
2021-05-16T22:45:15.256Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-000001-sesparse.vmdk" : closed.
2021-05-16T22:45:15.258Z| vcpu-0| I125: Checkpoint_Unstun: vm stopped for 52053 us
2021-05-16T22:45:15.258Z| vcpu-0| I125: SCSI: switching scsi0 to push completion mode
2021-05-16T22:45:15.259Z| vcpu-0| A100: ConfigDB: Setting scsi0:0.redo = ""
2021-05-16T22:45:15.259Z| vcpu-0| I125: DISK: OPEN scsi0:0 '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' persistent R[]
2021-05-16T22:45:15.259Z| vcpu-0| I125: DISKLIB-VMFS  : "/vmfs/volumes/hdd/vm-failed/vm-failed_1-flat.vmdk" : open successful (10) size = 68719476736, hd = 254855054. Type 3
2021-05-16T22:45:15.259Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: "vm-failed_1-flat.vmdk" (0xa)
2021-05-16T22:45:15.259Z| vcpu-0| I125: DISKLIB-LINK  : Opened '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' (0xa): vmfs, 134217728 sectors / 64 GB.
2021-05-16T22:45:15.259Z| vcpu-0| I125: DISKLIB-LIB_BLOCKTRACK   : Resuming change tracking.
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 254855054.
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISKLIB-CBT   : Successfuly created cbt node f30c78e-cbt.
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/f30c78e-cbt
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISKLIB-LIB   : Opened "/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk" (flags 0xa, type vmfs).
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISK: Disk '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' has UUID '60 00 c2 9e 1f 21 a6 ed-94 e2 7f 6d 0b 59 44 96'
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISK: OPEN '/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk' Geo (8354/255/63) BIOS Geo (0/0/0)
2021-05-16T22:45:15.260Z| vcpu-0| I125: Creating virtual dev for 'scsi0:0'.
2021-05-16T22:45:15.260Z| vcpu-0| I125: DumpDiskInfo: scsi0:0 createType=11, capacity = 134217728, numLinks = 1, allocationType = 1
2021-05-16T22:45:15.260Z| vcpu-0| I125: SCSIDiskESXPopulateVDevDesc: Using FS backend
2021-05-16T22:45:15.260Z| vcpu-0| I125: DISKUTIL: scsi0:0 : geometry=8354/255/63
2021-05-16T22:45:15.260Z| vcpu-0| I125: SCSIFilterESXAttachCBRCInt: CBRC not enabled or opened without filters,skipping CBRC           filter attach.
2021-05-16T22:45:15.260Z| vcpu-0| I125: SCSIFilterSBDAttachCBRC: device scsi0:0 is not SBD. Skipping CBRC attach SBD way.
2021-05-16T22:45:15.265Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=vim-cmd-e2-dac3 seq=705: Completed Snapshot request.
2021-05-16T22:45:15.265Z| vcpu-0| I125: Turning off snapshot info cache.
2021-05-16T22:45:15.265Z| vcpu-0| I125: Turning off snapshot disk cache.
2021-05-16T22:45:15.265Z| vcpu-0| I125: ConsolidateEnd: Snapshot consolidate complete: The operation completed successfully (0).
2021-05-16T22:45:15.265Z| vcpu-0| I125: Vix: [mainDispatch.c:4239]: VMAutomation_ReportPowerOpFinished: statevar=3, newAppState=1881, success=1 additionalError=0
2021-05-16T22:45:18.774Z| vcpu-0| I125: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/hdd/vm-failed/vm-failed_1.vmdk'
2021-05-16T22:45:18.774Z| vcpu-0| I125: DDB: "longContentID" = "ad37462b7a10b881f25c55a79f40922f" (was "9b885358f9a744640b58f1a5d2a2b710")
2021-05-16T22:45:18.777Z| vcpu-0| I125: DISKLIB-CHAIN : DiskChainUpdateContentID: old=0xd2a2b710, new=0x9f40922f (ad37462b7a10b881f25c55a79f40922f)

What can be done to replicate the VM vm-failed with cbt?

Best regards,
Peter

#70 General matters » XSIBackup-DC --replica=cbt: --options=R appears to be without function » 2021-05-16 22:35:49

herrep
Replies: 14

Hi,

when executing XSIBackup-DC 1.5.0.3 with action -replica=cbt for ctk-enabled VMs, --options=R appears to be without function.

This is the command I executed right after having enabled cbt for the VM named vm:

"/vmfs/volumes/hdd1/XSIBackup-DC/xsibackup" \
--replica=cbt \
"VMs(vm)" \
"root@nas:22:/volume1/backup" \
--options=R \
--exclude="data[-.].*vmdk" \
--check=full \
--quiesce \
--config-backup \
--mail-to="<mail-recipient>" \
--use-smtp="1" \
--description="replica hot" \
--backup-how=hot \
--remote-path="/usr/bin/xsibackup"

I would expect that the vmx file at the target disk contains a VM name that is appended by _XSIREP and that the replicated VM is registered at the ESXi host. In the past, I used a similar setup where XSIBackup-Pro OneDiff resulted in replicated VMs. The corresponding VM names were automatically appended by _XSIBAK and registered at the ESXi host. The only change was that I now use XSiBackup-DC instead of XSIBackup-Pro.

However, the expected behavior does not take place. The files of the VM get copied, but there is no registration and also not alteration of the replicated VM name with an appaended _XSIREP. The first run of the above-shown command shows the following output in xsibackup.log:


Performing --replica action
-------------------------------------------------------------------------------------------------------------
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
vm Hardware Version is: 14
-------------------------------------------------------------------------------------------------------------
All snapshots were removed, as vm is engaged in a CBT job
-------------------------------------------------------------------------------------------------------------
Virtual Machine Name: vm
-------------------------------------------------------------------------------------------------------------
Creating snapshot VM : vm (powered on)
-------------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-------------------------------------------------------------------------------------------------------------
Remote target dir is empty, full sync will be performed
-------------------------------------------------------------------------------------------------------------
Virtual Machine: vm
-------------------------------------------------------------------------------------------------------------
Backup start date: 2021-05-16T22:15:32
-------------------------------------------------------------------------------------------------------------
2021-05-16 22:15:32 | Backing up 29 files, total size is 96.46 GB
-------------------------------------------------------------------------------------------------------------
    NUMBER	                                                   FILE             SIZE          PROGRESS
-------------------------------------------------------------------------------------------------------------
    1/29                                                         vm.vmx          3.45 KB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------
    2/29                                      vm-flat.vmdk (CBT 1 full)         64.00 GB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------
    3/29                                                        vm.vmdk        583.00 B     | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    4/29                                                        vm.vmsd        540.00 B     | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    5/29                                                       vm.nvram          8.48 KB    | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    6/29                                                        vm.vmxf        150.00 B     | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    7/29                                                     vm-aux.xml        142.00 B     | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    8/29                                                 vmware-162.log        319.17 KB    | Done  66.35%
-------------------------------------------------------------------------------------------------------------
    9/29                                       vmx-vm-2040435808-2.vswp         94.00 MB    | Done  66.35%
-------------------------------------------------------------------------------------------------------------
   10/29                                                 vmware-161.log        244.02 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   11/29                                                 vmware-163.log        382.70 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   12/29	                                   vmx-vm-2040435808-1.vswp                    [open excluded]
-------------------------------------------------------------------------------------------------------------
   13/29                                                     vmware.log        220.79 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   14/29                                                 vmware-164.log        237.37 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   15/29	                                                 vm.vmx.lck                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   16/29                                                 vmware-165.log        254.22 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   17/29                                                 vmware-166.log        438.32 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   18/29                                                        vm.vmx~          3.44 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   19/29                                            vm-flat.vmdk___hash         82.00 B     | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   20/29	                                           vm-799e9460.vswp                    [open excluded]
-------------------------------------------------------------------------------------------------------------
   21/29	                                                vm-ctk.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   22/29                                                    vm.vmsd.tmp         45.00 B     | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   23/29                                                     vm.vmx.tmp          3.44 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   24/29                                            vm-Snapshot662.vmsn         28.25 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   25/29                                                vm.vmdk.extents          8.63 KB    | Done  66.44%
-------------------------------------------------------------------------------------------------------------
   26/29	                                    vm-000001-sesparse.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   27/29	                                             vm-000001.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   28/29	                                         vm-000001-ctk.vmdk                 [skipped excluded]
-------------------------------------------------------------------------------------------------------------
   29/29                                     vm-quiesce_manifest662.xml        120.00 B     | Done  66.44%
-------------------------------------------------------------------------------------------------------------
Total size:                                                                     64.09 GB    | Done 100.00%
-------------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-------------------------------------------------------------------------------------------------------------
Backup end date: 2021-05-16T22:22:43
-------------------------------------------------------------------------------------------------------------
Time taken: 00:07:11 (431 sec.)
-------------------------------------------------------------------------------------------------------------
Total time:      431 sec.
-------------------------------------------------------------------------------------------------------------
Full file speed:	                                                                   152.28 mb/s
-------------------------------------------------------------------------------------------------------------
Real data speed:	                                                                   163.36 mb/s
-------------------------------------------------------------------------------------------------------------
Item backup completed without errors
-------------------------------------------------------------------------------------------------------------

What needs to be done that _XSIREP is appended to the VM name of the replicated VM and that the replicated VM is registered?

Best regards,
Peter

#71 Re: General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-16 18:36:29

herrep

Ah, probably --remote-path does not only require the directory to xsibackup, but the full path?

--remote-path=/usr/bin/xsibackup did the trick, with xsibackup binary stored in /usr/bin.

#72 Re: General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-16 18:31:46

herrep

Thanks to your explanations I could add the keys correctly and login. But when running a replica action, I see the following result in the log file:

Performing --replica action
-------------------------------------------------------------------------------------------------------------
ash: /usr/bin: Is a directory
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
vm Hardware Version is: 14
-------------------------------------------------------------------------------------------------------------
Virtual Machine: vm
-------------------------------------------------------------------------------------------------------------
Backup start date: 2021-05-16T18:26:22
-------------------------------------------------------------------------------------------------------------
2021-05-16 18:26:22 | Backing up 17 files, total size is 10.08 GB
-------------------------------------------------------------------------------------------------------------
    NUMBER                                                       FILE             SIZE          PROGRESS
-------------------------------------------------------------------------------------------------------------
    1/17                                                          vm.vmx          2.80 KB    | Done   0.00%
-------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------
Removed current target dir: /volume1/backup/vm
-------------------------------------------------------------------------------------------------------------
SIGTERM (13) condition was trapped: check logs for more details
-------------------------------------------------------------------------------------------------------------
Cleaning up...
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------

The error log does not provide additional output.

#73 Re: General matters » XSIBackup-DC: SMTP error 550 when using option --subject » 2021-05-16 18:08:52

herrep

I already did this with no success. Therefore, I would rather like to analyze log files. But I do not know where to start and my trials to find a suitable log file via "find" where not successful...

#74 Re: General matters » XSIBackup-DC: tmp directories not removed after backup » 2021-05-16 18:06:47

herrep

Having the Synology NAS in the LAN besides the ESXi, do you prefer xsibackup executed on the ESXi with the Synology NAS accessed over NFS?

Or do you rather suggest to go for the server/client mode with xsibackup running on ESXi having an IP-based target where xsibackup is additionally running on the Synology NAS?

#75 General matters » XSIBackup-DC: Permission denied at Synology NAS after --add-key » 2021-05-16 13:46:01

herrep
Replies: 6

Hi,

I followed the [url=https://33hops.com/xsibackup-datacenter-running-as-server-in-synology-nas.html]guide[/url] to access my Synology NAS as IP target in a XSIBackup-DC --replica job.  Therefore, I started with --add-key which appeared to work fine:

[root@esxi:/vmfs/volumes/hdd/XSIBackup-DC] ./xsibackup --add-key root@nas:22
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.5.0.3: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------
PID: 34503796, Running job as: root
-------------------------------------------------------------------------------------------------------------
Key exchange routine
-------------------------------------------------------------------------------------------------------------
This is the routine to exchange keys with other servers
You will be prompted to enter the remote server's password
a number of times. Please do so, this is a one time operation
-------------------------------------------------------------------------------------------------------------
/!\ Remember to enable SSH out in the ESXi firewall
-------------------------------------------------------------------------------------------------------------
root@nas's password:
The remote server is of type: linux                                            OK
-------------------------------------------------------------------------------------------------------------
Let's look for the 'authorized_keys' file there...
-------------------------------------------------------------------------------------------------------------
root@nas's password:
The /root/.ssh/authorized_keys file does not exist
Do you want to create it? (y/n): y
root@nas's password:
The file /root/.ssh/authorized_keys was created
-------------------------------------------------------------------------------------------------------------
Checking if local public key exists at remote file...
-------------------------------------------------------------------------------------------------------------
root@nas's password:
Add local public key to root@nas:22:/root/.ssh/authorized_keys?
-------------------------------------------------------------------------------------------------------------
root@nas's password:
Local public key <xsibackup_id_rsa> has been
added to root@nas:22:/root/.ssh/authorized_keys                       OK
-------------------------------------------------------------------------------------------------------------
Remote server successfully linked for user root
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
SSH session was closed        OK
-------------------------------------------------------------------------------------------------------------
Removed PID                   OK
-------------------------------------------------------------------------------------------------------------

Then I started my replica --action:

[root@esxi:/vmfs/volumes/nas/XSIBackup-DC] etc/jobs/099
Performing a cold backup as per the --backup-how argument.
-------------------------------------------------------------------------------------------------------------
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.5.0.3: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------
License: | (c)XSIBackup-DC
-------------------------------------------------------------------------------------------------------------
Server root@nas:22 is not linked by public key, use --add-key command to exchange keys, returned: root@nas: Permission denied (publickey,password).
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Unlocked backup               OK
-------------------------------------------------------------------------------------------------------------
Removed PID                   OK
-------------------------------------------------------------------------------------------------------------
[root@esxi:/vmfs/volumes/nas/XSIBackup-DC]

When I manually log in at the Synology NAS from the ESXI server, I still need to enter my password, but ssh access to root@nas works.
xsibackup is in place at root@nas:/usr/bin.

When I re-run --add-key at the ESXI server, I cannot login any longer at root@nas. I see the key stored in /root/.ssh/authorized_keys at the Synology NAS, but I assume that something is wrong with this key, as I need to remove the key for the ESXI server therefrom to re-run --add-key successfully again.

However, I still fail to generate an authorized key that can be used for remote login from xsibackup.

Version: XSIBackup-DC 1.5.0.3

Best regards,
Peter

Pages: 1 2 3 4 5

Board footer