Implement domain log watching

Implements the ability for a client to watch almost-live domain
console logs from the hypervisors. It does this using a deque-based
"tail -f" mechanism (with a configurable buffer per-VM) that watches
the domain console logfile in the (configurable) directory every
half-second. It then stores the current buffer in Zookeeper when
changed, where a client can then request it, either as a static piece
of text in the `less` pager, or via a similar "tail -f" functionality
implemented using fixed line splitting and comparison to provide a
generally-seamless output.

Enabling this feature requires each guest VM to implement a Libvirt
serial log and write its (text) console to it, for example using the
default logging directory:

```
<serial type='pty'>
    <log file='/var/log/libvirt/vmname.log' append='off'/>
<serial>
```

The append mode can be either on or off; on grows files unbounded,
off causes the log (and hence the PVC log data) to be truncated on
initial VM startup from offline. The administrator must choose how
they best want to handle this until Libvirt implements their own
clog-type logging format.
This commit is contained in:
2019-04-11 19:06:06 -04:00
parent 5ad2dda6d4
commit b6ecd36588
8 changed files with 289 additions and 36 deletions

View File

@ -571,6 +571,37 @@ def vm_info(domain, long_output):
retcode, retmsg = pvc_vm.get_info(zk_conn, domain, long_output)
cleanup(retcode, retmsg, zk_conn)
###############################################################################
# pvc vm log
###############################################################################
@click.command(name='log', short_help='Show console logs of a VM object.')
@click.argument(
'domain'
)
@click.option(
'-l', '--lines', 'lines', default=1000, show_default=True,
help='Display this many log lines from the end of the log buffer.'
)
@click.option(
'-f', '--follow', 'follow', is_flag=True, default=False,
help='Follow the log buffer; output may be delayed by a few seconds relative to the live system. The --lines value defaults to 10 for the initial output.'
)
def vm_log(domain, lines, follow):
"""
Show console logs of virtual machine DOMAIN on its current node in the 'less' pager or continuously. DOMAIN may be a UUID or name. Note that migrating a VM to a different node will cause the log buffer to be overwritten by entries from the new node.
"""
# Open a Zookeeper connection
zk_conn = pvc_common.startZKConnection(zk_host)
if follow:
# Handle the "new" default of the follow
if lines == 1000:
lines = 10
retcode, retmsg = pvc_vm.follow_console_log(zk_conn, domain, lines)
else:
retcode, retmsg = pvc_vm.get_console_log(zk_conn, domain, lines)
cleanup(retcode, retmsg, zk_conn)
###############################################################################
# pvc vm list
###############################################################################
@ -1331,6 +1362,7 @@ cli_vm.add_command(vm_move)
cli_vm.add_command(vm_migrate)
cli_vm.add_command(vm_unmigrate)
cli_vm.add_command(vm_info)
cli_vm.add_command(vm_log)
cli_vm.add_command(vm_list)
cli_network.add_command(net_add)