2018年5月25日 星期五

[storage, extend partition] 如何用最簡單的方法擴充延展一個快滿的 VirtualBox partition (10 分鐘內)

How to extend a VirtualBox partition
井民全, Jing, mqjing@gmail.com
ETA: 10 min
Google doc: This document

1. Purpose

這份文件教你用最簡單的方法擴充你的 VirtualBox VM 磁碟空間.

2. Problem

VM 在使用後, 原本配置的空間不符使用, 如何在 10 分鐘內, 擴充這個 partition 的空間

3. Procedure

觀念上, 就是利用 vboxmanage 改變 vdi 的容量, 然後在利用 GParted 延展你的主要 partition.

3.1. 用 vboxmanage 更改 VDI 大小

Step 1: 先備份你的 VDI
cp xxx.vdi xxx.vdi.bak
Ex:


Step 2: 擴充你的虛擬硬碟空間
vboxmanage modifyhd 虛擬硬碟名稱.vdi -- resize 新大小(MB)
Ex: 設定延展為 100 GB
vboxmanage modifyhd MyLinux.vdi --resize 100000

驗證:  在 VM Setup 的 storage 欄位, 你會看到 Virtual Size 已經變成 100 GB.

接下來, 我們要來延展這個虛擬硬碟裡面的 partition

3.2. 重新調整這個 vdi 的 partition

Step 1: 下載處理 partition 的工具 GParted Live (here)
Ex:
Gparted-live-0.31.0-1-amd64.iso

Step 2: 建立 GParted VM 並且掛上你已經延展的 vdi 檔
  1. 建立 VM, 直接用 Gparted-live-0.31.0-1-amd64.iso 放在光碟
  1. 掛上已經擴展的 VDI
    1. 把你剛剛延展的 vdi 檔, 掛進來
    2. 開機


Step 3: 使用 GParted 來延展你原先的主要 partition
  1. 查閱目前 vdi 的 partition 狀況
  2. 移除 /dev/sda2 (extended) 和在裡面的 /dev/sda5 (swap)
這樣才能延展我們主要的 partition /dev/sd1
  1. 延展 /dev/sd1 partition 的大小
作法: 滑鼠選 [/dev/sda1] -> [Resize]
    1. 開始延展主要 partition /dev/sda1: 注意, 要留下 1023 MB 給 Swap 用, 所以 Free space following (MiB) 設定 1023, 然後, 拉一下上面的圖, 把其餘空間全部推給 /dev/sda1, 如下圖所示:
  1. 最後, 建立 swap partition 給你的 VM
    1. 建立 Extended Partition: 點選剩下 [Unallocated partition] -> [New] -> Extended Partition
    1. 建立 Swap File System: 滑鼠點選 Extended Partition 下面的 unallocated partition, 建立 swap File system
  1. 選擇 [Apply] 開始執行

Step 4: Shutdown the GParted VM


4. Verification


(Edit)

完成!

5. Reference


2018年5月21日 星期一

[morphology, dilation] 最簡單的方法做 grey morphology

How to use ndimage grey morphology
井民全, Jing, mqjing@gmail.com
Back to the Main Page
Google doc: This document

Purpose

This document describes how to use scipy to do the mathematical morphology dilation operator on wave signal.

Code

gray dilation

import scipy.ndimage as ndimage


#grey dilation
a = np.zeros(7, dtype = np.int)
a[2:5] = 1
a[4] = 3
print('a = ', a)

ndimage.grey_dilation(a, size = (3))
Result
a =  [0 0 1 1 3 0 0]
Out[16]:
array([0, 1, 1, 3, 3, 3, 0])

Binary dilation

# binary dilation
import numpy as np
import scipy.ndimage as ndimage

a = np.zeros(5)
a[2] = 1
print('a = ', a)

b = ndimage.binary_dilation(a).astype(a.dtype)
print('b = ', b)
Result
a =  [ 0. 0.  1. 0. 0.]
b =  [ 0. 1.  1. 1. 0.]

Wave Singal as 1D signal

# Global Import
import numpy as np
import sys
from numpy import genfromtxt

# Custom Import
sys.path.insert(1, r'./functions')  # add to pythonpath
from smooth import *
from myplot import *
from ecg_util import *
from detect_peaks import detect_peaks

# Global Variable
# Visual
scale = 2
FIGSIZE = (32*scale, 1.5*scale)


my_data = genfromtxt('../database/iems/iEMS/II.csv', delimiter=',')

plt1 = drawLine(1, my_data, 'b', FIGSIZE, False)
#plt1.show()

y = ndimage.grey_dilation(my_data, size = (5))
plt1 = drawLine(1, y, 'r', FIGSIZE, False)
plt1.show()

Reference


2018年5月11日 星期五

[hypervisor, main] 常用 Hypervisor 問題與最簡單答案

常用 Hypervisor 問題與最簡單答案
井民全, Jing, mqjing@gmail.com

Google doc: This Document

Table of Contents

1. KVM

用 virt-manager 啟動的 VM 可以出去, 也可以 ssh host <-> guest
  1. bios: Enable Intel Virtual Technology TRUE
  2. [vmware, nested kvm] Install KVM under vmplayer (view)
  3. [qemu, windows, software] Qemu for Windows (view)
  4. [qemu, windows, basic] Create, Install and Boot System using Qemu, Windows Qemu Manager 7.0 (view)

1.1. VM

  1. [qemu, create] How to create a base image and install Ubuntu 12.04 Server (view)
    1. [qemu, create] How to install Ubuntu server on KVM (view)
  2. [qemu, virtualbox, create] Create a QEMU/KVM VM from a virtualbox vdi image (view)
  3. [qemu, image] How to create a copy-on-write image from a template (view)
  4. [qemu, run] Run your VM from qemu with host forwarding (view)
  5. [libvirt, live migration] How to migrate your VM in live -- virsh (view)
  6. [libvirt, image import] How to import a qcow2 to virtual image storage pool  (virt-manager:: view) (virt-install:: view)
  7. [libvirt, create] How to create your virtual machines from iso -- virt-manager (view)
  8. [libvirt, import vm] How to import a VM to your virt-manager -- virt-manager (view)
  9. [vmbuilder, create vm] Create a new virtual machine using vmbuilder -- 直接建立 user, passwd (view)
  10. [libvirt, list] How to list the VM -- virsh -list  view)
  11. [libvirt, live migration] How to migrate your VM in live (view)
  12. [libvirt, internal snapshot] How to do the internal snapshot for your VM (view)
  13. [libvirt, snapshot restore] How to restore your VM from a snapshot -- virsh snapshot-revert (view)
  14. [libvirt, snapshot delete] How to delete your VM snapshot -- virsh snapshot-delete (view)
  15. [libvirt] How to manage your virtual machines -- virsh (view) -- draft
    1. Live snapshot (live disk mirror, live block operation)

1.2. Network

  1. [qemu, user networking] How to setup the user networking mode for your KVM (view)
  2. [qemu, scp] How to transfer a file to your guest virtual machine (view)
  3. [qemu, image, compression] How to compress your qcow2 image -- qemu-img convert -c (view) -- draft
  4. [qemu, vnc] How to run qemu with vnc (view)
  5. [qemu, network, user/nat] How to setup the simple user network mode for your VM (view)
  6. [qemu, network, user/nat, host forward] How to enable the host forward option for your VM (view)
  7. [qemu, network, user/nat, customized NAT] (more complexity: change guest ip, change host ip)
  8. [qemu, network, tap] How to connect your VM to a tap device (view)
  9. [qemu, network, tap, bridge, 1vm] How to connect single VM with a bridge using TUN/TAP model (view)
  10. [qemu, network, tap, bridge, 2vm] How to connect two VMs with a bridge using TUN/TAP model (view)
  11. [qemu, network, socket] How to connect your VM to a VLAN in socket mode (view) -- draft
  12. [qemu, network, multicast] How to use multicast (view) -- draft
  13. [libvirt, network, redirection] How to run your VM with port redirection (view)
  14. [libvirt, network] guest -> host and host -> guest -- virt-manager (view)

1.3. Disk I/O

  1. [libvirt, disk performance] How to evaluate the VM I/O (ref)

1.4. Questions

  1. boot VM from remote qcow2 image, how?
  2. access guest image disk, how? guestfish (ref)

1.5. References

  1. [ref] A lot of reference document from IBM (ref)
  2. [mac os, qemu] Run Mac OS X Yosemite (ref)

2. Vagrant

  1. [vagrant, lab: 2-node with 500G] os: ubuntu/trusty64, hd: 500G, ip: 192.168.50.10/11 (download)
  2. [box, searh box] searh the box -- https://atlas.hashicorp.com/search
  3. [box, crate box] How to create a vagrant box (view) (go)
  4. [box, list box] How to list your vagrant box -- vagrant box list (view)
  5. [box, add box] How to install a box for your vagrant -- vagrant box add  (view)
  6. [box, copy box] How to copy your vagrant box -- just cp (view)

2.1. VM

  1. [vm, create vm] How to create your vm by vagrant -- vagrant init; vagrant up
  2. [vm, two vm] How to create two vms by vagrant (view)
  3. [vm , create vm] How to create vms using for loop (view)
  4. [vm, list vm] How to list your running vm -- vagrant status(view)
  5. [vm, destroy vm] How to destroy a VM -- vagrant status; vagrant destroy
  6. [vm, clean & restart] Clean & Restart a VM -- vagrant destroy;vagrant up
  7. [vm, hostname] How to setup hostname for your VM, vagrant (view)
  8. [vm, memory] How to setup memory for your VM, vagrant (view)

2.2. Network

  1. [network, bridge] How to setup a bridge device for your VM, vagrant (view)
  2. [network, private] How to setup a private network for your VM, vagrant (view)
  3. [network, nic] How to assign your vagrant network to specific network adapter (view)

2.3. Disk

  1. [disk, add new disk] How to add a new disk for your VM (view)
  2. [disk, multi config] How to create disk with random name (view)

2.4. Provision

  1. [provision, bash] How to use bash to provision your vm (view)
  2. [provision, bash, library file] How to use bash to provision your vm with library (view)
  3. [provision, puppet] The experimental environment as code by vagrant + puppet (view)
  4. [vagrant, show gui] How to show your vm in GUI -- vbox.gui = true (view)
  5. [vagrant, snapshot] How to enable vagrant snapshot (view)
  6. [vagrant, run script] How to run script from vagrant command -- vagrant ssh [your vm name] -- 'command 1;command2'
  7. [VagrantFile, pass parament] How to pass argument to your provision script in vagrant (view) (ref)
  8. [default password] What is the default username/password of vagrant? -- vagrant/vagrant
  9. [insecure key] How to keep using default insecurity key to your guest VM (view)
  10. [ssh, private key] How to use vagrant user' private key to login (view)
  11. [cli] How to run command in VM -- vagrant ssh {VM-NAME} -c "command"
  12. [add user] How to add deploy script with adding user (view)

2.5. Error Handling

  1. [error] A VirtualBox machine with the name 'ubuntu-xenial-16.04-cloudimg' already exists. (view)

3. VirtualBox

  1. [install] How to indatll VirtaulBox (view)
  2. [installation, Guest Additional] How to install VirtualBox Guest Additional Package (view) -- require gcc.

3.1. Cli

  1. [cli, vm, creat] How to create VM (view) (view2)
  2. [cli, vm, list How to list VM -- vboxmanage list vms
  3. [cli, vm, list running vm] How to list running vm -- vboxmanage list running VMs (view)
  4. [cli, vm uuid] How to get the vm uuid (view)
  5. [cli, vm info] How to get vm information -- vboxmanage guestproperty enumerate <vm-name>
  6. [cli, network,, ip] How to list running vm name and ip** (view)
  7. [cli, network, add bridge nic] How to create VM with a bridge nic (view)

3.2. File I/O

  1. [file, share] Share folder to VirtualBox (view)
  2. [file, usb] Enable USB device for your VirtualBox (view)
  3. [file, extend partition] How to extend a partition for Ubuntu VirtualBox (view)
  4. [file, mount] automatically mount a VirtualBox shared folder for guest OS Ubuntu (view)

3.3. Network

  1. [network, nat porting] Configuring port forwarding with NAT (view)
  2. [network, host-only, dhcp] How to disable DHCP server in host-only mode (view)
  3. [network, network, nat] Setup NAT, Can two NAT VMs communication? -- no (view)
  4. [network, private, Host-only] 建立自己的 Private Host-Only Network, 使用自己建立的 Host-only 專用虛擬網卡  (view)
  5. [network, private, Bridge] 建立自己的 Private Bridge Network, 使用自己建立的虛擬網卡 (view)
  6. [network, private network] Openvswitch vs. VirtualBox, http://networkstatic.net/open-vswitch-on-virtualbox/
    • 觀念:  讓 VM 連接到聯外的網路,
      • 利用 ip 指令建立 tap 型態的虛擬 nic => vboxnet01, 然後你就可以讓 virtualbox 連到它 (ref).
      • 要出去 => 用 ovsctl 建立一個新的 bridge, 把 虛擬 nic  和 實體 nic eth0 都接上去. 指定 br 外網的 ip.
    • 觀念: 建立另一個 private network
      • 建立 virtual nic, 讓 VM 用 host-only 連到這個 nic.
      • 每一台 VM 都用同樣的方式, 連到相同的 nic
  7. [network, vlan, concept] {Host-Only (vb) == Private Network (vagrant) == VLAN} via eth0 (NAT) connect to Internet
  8. [network, bridge, virbr0] What is virbr0 (view)

3.4. VM

  1. [vm, clone] How to clone a virtualbox vm (view)
  2. [vm, linux-02] How to use linux-02 image as you just enough VM in VirtualBox (view)

3.5. Misc

  1. [openstack, fuel] How to install openstack on virtualbox (view), (new)
  2. [display, dpi] How to adjust the guest display dpi
    • Vitualbox -> (select guest) -> Settings -> Display -> Scale Factor



4. VMWare

  1. [serial] vSphere 5.5 serial number (view)
  2. [package, download] Evaluate and Download (go)

4.1. VMPlayer

  1. [vmplayer, install] Install VMPlayer (view)
  2. [vmplayer, install vmware tool] How to install vmware tool (view)
  3. [vmplayer,share] Share folder setting for VMware (view)
  4. [vmplayer, clone] How to copy your vmplayer image (go)
  5. [vmplayer, cp] scp vs. cp in VMware (view)
  6. [vmplayer, promiscuous] How to enable promiscuous mode? (view)
  7. [vmplayer, network] How to create vmware subnet? (view)
  8. [vmplayer, adapter] How to create vmware virtual adapter (view)

4.2. vSphere Client

  1. [vsphere web client, IE security] How to disable IE protected mode for vSphere Web Client (view)
  2. [vsphere client, display] How to modify the display resolution for your vm from vsphere client (view)
  3. [vsphere client, boot] How to setup VM to boot from CD-ROM (view)
  4. [vsphere web client, localized language] How to change the localized language? -- https://localhost:9443/vsphere-client/?locale=en_US# (view)
  5. [vSphere client] How to connect to the local vCenter: hostname=local, username=administrator@vsphere.local, password =xxx (view)

4.3. ESXi

  1. [esxi, install] How to install VMware hypervisor? (view)
    • 請先用 parted tool 或 ESXi partedUtil  delete 指令. 把硬碟中的 partition 全部清掉
    • usb stick installation Windows: lili with fat32 format
    • vSphere 5.5 requirement (view), OS support (view)
  2. [esxi, install, customization] How to create a customized iso for installing vSphere 5.5 (ref) (ESXi-Customizer)
  3. [esxi, install, usb] Install ESXi on USB driver (ref)
    • 請先用 parted tool 或 ESXi partedUtil  delete 指令. 把硬碟中的 partition 全部清掉
    • But you cannot use the USB stick as a datastore.
  4. [esxi, reset] How to reset the ESXi System (ref)
  5. [esxi, installation, pxe] How to install ESXi by PXE (draft)
  6. [esxi, 64bits simulation] How to enable 64-bit VM -- create vm and set the vcpu support Hardware Virtualization (view)
  7. [esxi, check nested VM]  Checking nested VM on ESXi (view)
  8. [esxi, hardware virtualization] How to enable hardware virtualization (view)
  9. [esxi, nested esxi] How to setup a nested ESXi lab at home (view)
  10. [esxi, list partition] How to list partitions from a disk (view)
  11. [esxi, delete partition] How to delete a partition from your storage device in ESXi (view)
  12. [esxi, clean up disk] How to clean up your disk before you install ESXi or VSAN (view)
  13. [esxi, get core dump partition] How to get the core dump partition info (view)
  14. [esxi, hdd -> ssd] How to simulate hdd -> ssd (draft)
  15. [esxi, ntp server] Let an esxi host as a ntp server (ref)
  16. [esxi, ntp client] Assign esxi host as a ntp client (ref)
  17. [esxi, vmdk] How to extend size of VMDK (view)
  18. [esxi, hba driver] How to install HBA driver (view)
  19. [esxi, console] How to login the ESXi host from console (view)
  20. [esxi, disk usage] How to determine disk usage in thin provision (view)

4.4. vCenter

  1. [vcenter, install, package] How to install vcenter 5.5 (view) (usb ref)
  2. [vcenter, install, appliance, 5.5] How to install vcenter by vcenter server appliance, port: 9443  (view)
  3. [vcenter, install, appliance, 6.0] How to install vcenter by vcenter server appliance, port: 9443  (view)
  4. [vcenter, service] How to check the vcenter service is ready (view)
  5. [vcenter, troubleshooting] vcenter cannot login (view)
  6. [vcenter, change ip] How to change vcenter IP from appliance, 5.5 (view)
  7. [vcenter, enable vMotion] How to enable vMotion (view)
  8. [vcenter, enable vSphere HA] How to enable vSphere HA (slide)
  9. [vcenter, DRS] How to enable VMware Distributed Resource Scheduler (slide, draft)
  10. [vcenter, vm template, register] How to register VM from datastore (view)
  11. [vcenter, create vm, template] How to create a new vm from template (view)
  12. [vcenter, add datastore, nas] How to add a new datastore from NFS (view)
  13. [vcenter, startup] How to setup the default VM startup and shutdown automatically (view)
  14. [vcenter, alarm, definition] Where is the alarm definition (view)
  15. [vcenter, storage, vm size] How to evaluate a VM size (view)
  16. [vcenter, storage, thin provision] How thin provisioning works (slide), 11 pages
  17. [vcenter, storage, provision] How to check the VM or Virtual Disk provision type (view)
  18. [vcenter, storage, rdm] How to create RDMs for your SATA device (view)
  19. [vcenter, storage, iscsi] How to setup iSCSI device for your ESXi system (view)
  20. [vcenter, switch, jumbo] How to setup jumbo frame for your kernel vswitch (view)

4.5. VSAN

  1. [vsan, summary] the summary of vsan (view)
  2. [vsan, white paper] the data locality whitepaper for VSAN (ref) (draft)
  3. [vsan, lab] The VMware virtual SAN lab (ref)
  4. [vsan, installation] How to enable vsan for your infrastructure (view)
  5. [vsan, remove] How to remove vsan from your system (view)
  6. [vsan, location] Where is your data location in vsan enabled (view)
  7. [vsan, storage policy] How to setup your vm policy for vsan storage -- concept, stripe number, tolerate  (view)
  8. [vsan, evaluate] How to evaluate the stripe number for your vm storage (view)
  9. [vsan, tool, check disk] How to check your storage device -- HD Tune Pro
  10. [vsan, cache status] How to determine the vsan cache status -- write through/write back, hit rate? (view)
VM
  1. [vm, vmx file] Where is the vm configuration file (vmx file) for a vm (view)
  2. [vm, powercli, set locale) How to setup the locale for your powerCli (view)
  3. [vm, start, dos] How to start a VM (view)
  4. [vm, copy&paste] How to enable copy & paste (view)
  5. [vm, add hd] How to add new storage from vSphere Web Client (view)
  6. [nsx, introduction] Introduce to the VMware NSX (draft)
  7. [nsx, installation] How to install NSX (draft)
  8. [vmmark] Introduce to the VMMark (slide draft)

4.6. I/O

  1. [direct io, pass-through] How to setup passthrough for a PCI device (view)

4.7. Misc

  1. [vmtool] How to install vmtool in your VM (view)
  2. [vmware, tool] How to install vmware tool in your Ubuntu -- extract the tgz. run the pl (ref)
  3. [evo rail, hw spec] What is the hardware spec for vmware evo rail (view)
  4. [vds, create] How to create vSphere distributed switch (view)
  5. [vds, create kernel switch] How to create a VMKernel switch on VDS (ref)

References
  1. [vDS] Distributed Virtual Switches (draft)
  2. [vds, config] Configuring a vSphere Distributed Switch in the vSphere Web Client (ref)
  3. [vds, config] Configuring vNetwork Distributed Switch using vCenter Server (1010557) (ref)
  4. [vds, overview] Overview of vNetwork Distributed Switch concepts (ref) (6.0)
  5. [vds, lacp] LACP Support on a vSphere Distributed Switch (ref)
  6. [esxi, deployment] Completely Automated Deployment ESXi (ref)
  7. [esxtop] Field of esxtop (ref)

Questions
  1. What is the dirty ratio of VSAN?

5. VMWare Cli

5.1. ESXCLI

  1. [esxcli, installation] vCLI, vSphere Command Line Interface (view)
  2. [esxcli, create vm] How to create a VM (draft)
  3. [esxcli, list vm] How to list the running virtual machine on the system (view)

5.2. PowerCLI

  1. [powercli, installation] Download the PowerCli (5.5_download) (5.5 r2 download, release note) (6.0 download) (document)
  2. [powercli, init] Prepare your powercli environment (view)

5.2.1. VM

  1. [powercli, list vm] List VM by PowerCli (view)
  2. [powercli, create vm] Create VM by PowerCli (view)
  3. [powercli, create vm + 3 nic] How to create VM with 3 NIC (view)
  4. [powercli, create vm, array] How to create VM from array by PowerCli (view)
  5. [powercli, create 10 vm + csv] How to batch create VM (view)
  6. [powercli, clone vm] How to clone to VM (view)
  7. [powercli, create vm, template] How to create a VM from template (view)
  8. [powercli, start vm] How to start VM by PowerCli (view)
  9. [powercli, stop vm] How to shutdown and power off the vm by PowerCli (view)
  10. [powercli, remove vm] How to remove VM by PowerCli (view)
  11. [powercli, move vm] How to move VM by PowerCli (view)

5.2.2. Storage

  1. [powercli, add cd] How to add CD device to your VM (view)
  2. [powercli, add hd] How to add a storage to your VM (view)
  3. [powercli, check, storage policy] How to check the storage policy for VM and Disk (view)
  4. [powercli, setup, storage policy] How to setup a specified storage policy to VM and Disk (view)
  5. [powercli, setup, storage format] How to setup the Disk storage format to eager zero (view)
  6. [powercli, add hd to controller] How to add a lot of hdd to your VM and attached them to the specified controller (view)
  7. [powercli, remove hd] How to remove the second hd from your VM (view)
  8. [powercli, list hd detail] How to list hard disk detail (view)

5.2.3. Snapshot

  1. [powercli, snapshot list] How to get snapshot list from you VM (view)

5.2.4. Network

  1. [powercli, network traffic] How to get esxtop network traffic using get-esxtop command (view)
  2. [powercli, 60 sec network traffic] How to retrieving 60 sec network traffic from esxtop (view)

5.2.5. Misc

  1. [powercli, cpu num] How to get cpu number from esxtop (view)
  2. [powercli, lcpu used] How to get lcpu used (view)
  3. [powercli, virtualportgroup] How to create a virtual port group by PowerCli (view)
  4. [powercli, run dos batch file] How to run the DOS batch file (view)
  5. [powercli, vm ip] How to get the VM IP from powercli (view)
  6. [powercli, remote execute bat] Run remote batch file where is located in a share folder (view)
  7. [powercli, computer name] An example on automatically changing VMs computer name (view)
  8. [powercli, wait for task complete] How to wait for all tasks to finish (view)

6. Mesos

  1. Mesos (ref)

7. Hyper-V

  1. [vm, create] How to create a Hyper-V VM (view)
  2. [vm, internet] How to setup your VM on Internet (view)