Scripting And Integration With OVirt

1y ago
2.39 MB
120 Pages
Last View : 4d ago
Last Download : 7m ago
Upload by : Kamden Hassan

Scripting And Integration with oVirtOved Ourfali, [email protected] Software Engineer, Red-HatCloudOpen Europe - October 20131

AgendaPart 1 – REST-based APIsIntroductionoVirt APIoVirt Shell (CLI) (Demo)oVirt SDKDeltacloud APIsPart 2 – Extension APIsUI Plugin API (Demo)Scheduling APIVDSM hooks2

Part 1REST-based APIs3

What can I do with it?Infrastructure configurationHost configuration and managementNetwork and storage configurationVirtual machine (VM) configuration and managementNetworking for the Guest, Virtual disks, VM propertiesUser managementDaily maintenance and VM lifecycle managementAdvanced operations not available in the GUIAnd much more .4

Where would I use it?Scripting and utilitiesA way to integrate with other software used in theorganizationAutomation of tasks – provisioning and managementSoftware development – of specific add-ons orapplicationsPerformance monitoring and reports5

API methodsRESTSDK (Python/Java)Shellhttps://host:port/api/vms api.vms.list()list vmsReturns:Returns:XML/JSON/. vm id "aee0dbce-1591-44d4-9052-c2209b3e45b8"href "/api/vms/aee0dbce-1591-44d4-9052c2209b3e45b8" name Austin /name actions link rel "shutdown" href down"/ link rel "start" href t"/ .Returns:list of VM objectsFormatted textid: 18df94a7-048f-43069cfd-a74e8ea3b907name: Bostondescription: Main service forBostoncluster-id: res:2cpu-topology-sockets : 16

What method to use?Depends onWho you are and what is your role?What you are trying to do?What is the level of your expertise?Few examplesSystem Administrator who is managing small virtualizedservers, using the Shell interface for a quicktroubleshootDatacenter admin, running large virtualizedenvironment, using the SDK to automate provisioningand managing hundreds of virtual machinesSoftware developer who integrates creating his ownlibraries over REST API7

API ConceptsAll APIs integrate through the oVirt engineAll types of APIs are based on the web servicesinterfaceREST as the coreSDK on top of RESTShell implemented on top the Python SDKBackward and forward compatibilitySecure accessSession-based access8

oVirt API9

HTTP methods in RESTGETRequests a representation of the specified resource. Requestsusing GET (and a few other HTTP methods) "SHOULD NOThave the significance of taking an action other than retrieval."POSTSubmits data to be processed to the identified resource. Thedata is included in the body of the request.PUTUploads a representation of the specified resourceDELETEDeletes the specified resource10

Media typesXML vms vm id ”xxx” name yyy name /vm /vms JavaScript Object Notation (JSON){}“vms” : [“vm” : {“id” : ”xxx”,“name” : ”yyy” } ]11

oVirt-API URI structurehttp(s)://server:port/api/vms/vm id/disks/disk id12345671. protocol2. server endpoint details3. entry point (base resource)4. collection5. resource6. sub-collection7. sub-resource12

oVirt-API Resource Structure vm id "xxx" href "/api/vms/xxx" identification details name vm1 iscsi /name status DOWN /status memory 10737418240 /memory cpu topology cores "1" sockets "1"/ /cpu start time 2011-07-13T12:05:34.931Z /start time creation time 2011-05-31T16:47:51 03:00 /creation time actions link rel "start" href "/api/vms/xxx/start"/ link rel "stop" href "/api/vms/xxx/stop"/ /actions link rel "disks" href "/api/vms/xxx/disks"/ link rel "nics" href "/api/vms/xxx/nics"/ cluster id "zzz" href "/api/clusters/zzz"/ template id "yyy" href "/api/templates/yyy"/ /vm MetadataResourcedetailsActionsLinks to relatedresources13

oVirt-API How-to (the methods)To list all collection resources, use GETGET http(s):/server:port/api/vmsTo retrieve specific resource, use GETGET http(s)://server:port/api/vms/{ID}To create a resource, use POSTPOST http(s)://server:port/api/vms vm . /vm To update the resource, use PUTPUT http(s)://server:port/api/vms/{ID} vm name new name /name /vm To remove the resource, use DELETEDELETE http(s)://server:port/api/vms/{ID}14

Clients / ToolsAny HTTP library/client can be used as aa client for oVirt-APICommon used clients areFirefox REST ClientChrome REST-ClientLinux: curl / wget.15

Example - GETGET http(s)://server:port/api/vms/{ID}curl v u "[email protected]:password" H "Content type: application/xml" X GEThttp(s)://server:port/api/vms/{ID}16

Example - CREATECreate VMPOST http(s)://server:port/api/vms vm name my new vm /name cluster id "xxx" / template id "yyy" / /vm curl v u "[email protected]:password" H "Content type: application/xml" d ' vm name my new vm /name cluster name cluster name /name /cluster template name template name /name /template /vm ' 'http(s)://server:port/api/vms'17

Example - UPDATEUpdatePUT http(s)://server:port/api/vms/xxx vm name new name /name /vm echo " vm name new name /name /vm " /tmp/upload.xmlcurl v u "[email protected]:password" H "Content type: application/xml" T 18

Example - DELETEDeleteDELETE http(s)://server:port/api/vms/xxxcurl v u "[email protected]:password" X DELETEhttp(s)://server:port/api/vms/xxx19

RSDL - RESTful Services DescriptionLanguageDescribes parameter constraintsEasy way to understandHow to create the resourceWhat actions are available on a collectionWhat parameters to passMandatory/optional/read-onlyTypeOverloads20

RSDL link description (http(s)://engine:port/api?rsdl)21

Additional FunctionalityUser-level APIoVirt version 3.2 and above support user-level APIIn order to use it, you need to pass the “filter” HTTPheader, with a value of “true”Session supportWe use cookies to allow using the REST API withouthaving to re-authenticate on every requestSee more details ment22

oVirt ShellCommand Line Interface23

oVirt ShellConceptsInteractive shellUse tab for auto-completionUse arrow keys to browse previous commandsAbility to execute scriptsPipe output into shell commands24

oVirt Shell - Smart helphelp is dynamically created for each commandhelp command [arguments] [options]Help commandhelp addHelp command argumenthelp add storagedomainOr if it's for operation on a subcollection contexthelp add storagedomain --datacenter-identifier yyy25

oVirt Shell - ConnectWhen running the ovirt-shell, you'll get a disconnectedpromptTo connect as an administrator, run[oVirt shell (disconnected)]# connect --url "http://server:port/api"--user "[email protected]" --password "password"and as a user run[oVirt shell (disconnected)]# connect --url "http://server:port/api"--filter --user "[email protected]" --password "password"then you should get the “connected” prompt[oVirt shell (connected)]#26

oVirt Shell - Smart auto-completion27

oVirt Shell – Querying for Resourceslist resource Provides a list of these resources, minimal detailslist resource --show-allProvides a list of these resources, full detailslist resource --kwargs "param val"list resources using client side filteringlist vms --kwargs name p*list resource --query "param val"list resources using oVirt query engine filteringEtc.28

oVirt Shell – Resource ManipulationaddupdateremoveoVirt Shell – ActionsAction on resourceaction resource name action Action on sub-resourceaction sub-resource name action -- resource identifier name 29

oVirt Shell – Help on Actions30

oVirt Shell - Actions31

oVirt Shell - ExamplesAction with parametersaction host Atlantic install --root password 123456Create and updateadd vm --name BaseRHEL6 --memory 1073741824--template-name Blank --cluster-name Defaultupdate vm BaseRHEL6 --os-type rhel 6x64add nic --vm-identifier BaseRHEL6 --name nic1 –networkname ovirtmgmt32

oVirt Shell - ExamplesCreate and attach Diskscreate disk --size 10737418240 --wipe after delete False--format raw --sparse True --interface virtio--storage domains-storage domain"storage myImageStore" --bootable Truecreate disk --id r BaseRHEL6Orcreate disk --size 10737418240 --wipe after delete False--format raw --sparse True --interface virtio--storage domains-storage domain"storage myImageStore" --vm-identifierBaseRHEL633

oVirt Shell - ExamplesCreate Templatecreate template --vm-name BaseRHEL6 --nameBaseRHEL6Create VM from Templatecreate vm --template-name BaseRHEL6 --name myRHEL--cluster-name DefaultRemove VMremove vm my new vm34

oVirt SDK35

oVirt SDKMainly used for integration or advanced automationObject orientedCurrent bindingsJavaPythonLibgovirt (GObject wrapper for the oVirt REST API)Rbovirt – ruby binding for the oVirt REST APISlides will demonstrate the python SDKJava - - -

oVirt SDKConceptsComplete protocol abstractionFull compliance with the oVirt API architectureAuto-completionSelf descriptive, intuitive and easy to useAuto-generated37

oVirt SDK - Examples- Creating the proxyAdditional options:filter True for user-level API (default is False – admin API)persistent auth True for using API REST sessions (default is False)- Listing all collections- Listing collection'smethods.38

oVirt SDK - Examples- Querying collection with thesearch engine.- Querying collection by customconstraint.- Querying collection for specificresource.- Accessing resource methods andproperties.39

oVirt SDK - Examples- Accessing resource properties and sub-collections- Accessing sub-collection methods40

oVirt SDK - Examples- Creating a VM resource- Creating a Disk resource- Attach a disk to a VM, and activate it41

oVirt SDK - Examples- Querying sub-collection by custom constraint.- Retrieving sub-collection resource.- Accessing sub-collection resource properties and methods.42


DeltacloudOpen source Apache projectAbstracts the differences between cloud providersSupports Deltacloud, EC2 and CIMI APIsSupports many cloud providersEC2oVirtEucalyptusOpenNebulaOpenStack.44

MotivationHeterogeneous cloud and virtualization environmentExisting software working with common cloud APIs likeEC2Synaps – CloudWatch implementation over EC2 APIHeat (Openstack project)CloudFormation and CloudWatch supportAutomated scripts.45

Cloud APIsDMTF CIMI APICloud Infrastructure Management Interface 1.0.1 was published in October 2012Still new API, but aims to be the cloud standard APIEC2 API – Amazon Elastic Cloud APIDeltacloud API46

DeltacloudCIMI APIEC2 APIDeltacloud APIDeltacloud ServerEC2 driveroVirt driverEucalyptus driverAmazon EC2oVirtEucalyptus47

Deltacloud Links and Further ReadingDeltacloudDeveloper mailing list org#deltacloud on FreenodeMy blog with some useful examples on top of oVirthttp://ovedou.blogspot.com48

Part 2Extension APIs49

AgendaPart 1 – REST-based APIsIntroductionoVirt APIoVirt Shell (CLI) (Demo)oVirt SDKDeltacloud APIsPart 2 – Extension APIsUI Plugin API (Demo)Scheduling APIVDSM hooks50

UI PluginsCommand Line Interface51

Web Admin user interfaceExtend oVirt Web Admin user interface52

Web Admin user interface53

Web Admin user interface54

Web Admin user interface55

Web Admin user interface56

What's currently possible57

Writing plugins !DOCTYPE html html head !-- Fetch additional resources if necessary -- script type "text/javascript" src "jquery-min.js" /script !-- Actual plugin code -- script // Access plugin API from iframe contextvar api parent.pluginApi('myPlugin');// Register plugin event handler functionsapi.register({UiInit: function() {api.addMainTab('Foo Tab', 'foo-tab', '');}});// Tell plugin infrastructure that we are readyapi.ready(); /script /head body !-- HTML body is intentionally empty -- /body /html 58

UI plugin basicsPlugin host pageHosts actual plugin code (JavaScript)/usr/share/ovirt-engine/ui-plugins/ resourcePath / hostPage .htmlPlugin descriptorMeta-data default configuration/usr/share/ovirt-engine/ui-plugins/ descriptorName .jsonPlugin user configurationOverride default configuration, tweak runtime behavior/etc/ovirt-engine/ui-plugins/ descriptorName -config.json59

Supported API functionsaddMainTab(label, historyToken, contentUrl)addSubTab(entityTypeName, label, historyToken, contentUrl)setTabContentUrl(historyToken, contentUrl)setTabAccessible(historyToken, tabAccessible)StringBooleanNumberObject60

Supported API functionsaddMainTabActionButton(entityTypeName, label, buttonInterface,options)Can add the button at the toolbar, context menu, or bothaddSubTabActionButton(mainTabEntityName, subTabEntityName,label, buttonInterface, options)showDialog(title, contentUrl, width, mberObject61

Supported API ])UserLogin(fullUserName, d)Using this one you can do anything I showed earlierYou can also use the new external events feature to get amore native integrationMessageReceived (allows Plugin HTML to interact with the UIplugin)StringBooleanNumberObject62

Plugin descriptorMeta-data default configuration/usr/share/ovirt-engine/ui-plugins/ descriptorName .json{// A name that uniquely identifies the plugin (required)"name": "foo",// URL of plugin host page that invokes the plugin code (required)"url": "/webadmin/webadmin/plugin/foo/start.html",// Default configuration object associated with the plugin (optional)"config": { "band": "ZZ Top", "classic": true, "score": 10 },// Path to plugin static resources (optional)// Used when serving plugin files through PluginResourceServlet// This path is relative to /usr/share/ovirt-engine/ui-plugins"resourcePath": "foo-files"}63

Plugin user configurationOverride default configuration, tweak runtime behavior/etc/ovirt-engine/ui-plugins/ descriptorName -config.json{// Custom configuration object associated with the plugin (optional)// This overrides the default plugin descriptor configuration, if any"config": { "band": "AC/DC" },// Whether the plugin should be loaded on WebAdmin startup (optional)// Default value is 'true'"enabled": true,// Relative order in which the plugin will be loaded (optional)// Default value is Integer.MAX VALUE (lowest order)"order": 0}64

Runtime plugin configurationMerge user configuration (if any)on top of default configuration (if any){ "band": "ZZ Top", "classic": true, "score": 10 } { "band": "AC/DC" } { "band": "AC/DC", "classic": true, "score": 10 }65

Main steps in plugin development(1) Write plugin descriptor(2) Write plugin host page(3) See plugin in action66

UI Plugin - Live s/UIPlugins#UI Plugins d Line Interface67

Plugin Initialization68

Opening a Dialog69

Event Registration70

Event Registration – Cross-WindowCommunication71

Cross-Window Communication72

Let's start!Call api.ready() to tell the oVirt engine to startinitializing your plugin73

oVirt-Foreman plugin74

oVirt Plugin – Dashboard75

oVirt Plugin – VM Foreman Details76

oVirt Plugin – VM Foreman Graphs77

oVirt Plugin – start.html fileRegister section - register the following event handlersUiInit – add the main/sub tabs78

oVirt Plugin – start.html fileRestApiSessionAcquired – relogin to ForemanVirtualMachineSelectionChange – set sub-tabs URL ifthe selected VM exists in Foreman79

oVirt Plugin – start.html fileUserLogout – logout from Foreman80

oVirt Plugin – start.html fileHelper functions81

Scheduling APICommand Line Interface82

IntroductionThe need - construct user-defined scheduling policyRe: [Users] How to define max number of running VMs on a host? .I have 4 graphic workstations with 3 graphic cards on each. I wannapassthrough graphic cards to the VMs one by one, since one workstationhas only 3 cards, I must limit the number of running VM on a host to 3.83

Previous Scheduling MechanismExecutes the selected distribution algorithm on the ClusterEvenly DistributedPower SavingSchedulingSelects a host to run/migrate VMLoad balancingSelects a VM to migrate and Host to migrate toTwo distribution algorithms, taking into consideration onlyCPU usageNo way to construct user-defined policy84

New Scheduling MechanismHost 1Host 1Host 2Host 3Host 2Host 4Host 2Host 2Host 4Host 4func 1func 2sumFactor52Host 210254Host 431239**Host 4 sum: 3*5 12*2 39Host 485

Filter ModuleHost 1Host 1Host 2Host 3Host 2Host 4Host 2Host 2Host 4Host 4func 1func 2sumFactor52Host 210254Host 431239**Host 4 sum: 3*5 12*2 39Host 486

Weight ModuleHost 1Host 1Host 2Host 3Host 2Host 4Host 2Host 2Host 4Host 4func 1func 2sumFactor52Host 210254Host 431239**Host 4 sum: 3*5 12*2 39Host 487

Filter ModuleLogical unit which filters out hostsClear cut logicEasy to write and maintainChained up-dependently to allow complete filteringAllows custom parametersExisting logic (pin-to-host, memory limitations, etc.) istranslated into filtersExternal filters written in python can be loaded intoengine88

Let's go back to the exampleRe: [Users] How to define max number of running VMs on a host? .I have 4 graphic workstations with 3 graphic cards on each. I wannapassthrough graphic cards to the VMs one by one, since one workstationhas only 3 cards, I must limit the number of running VM on a host to 3.Filter: filters out hosts with number running of vms 389

Filter Example90

Filter Example91

Filter Example92

Filter Example93

Filter Example94

Weight ModuleHost 1Host 1Host 2Host 3Host 2Host 4Host 2Host 2Host 4Host 4func 1func 2sumFactor52Host 210254Host 431239**Host 4 sum: 3*5 12*2 39Host 495

Weight ModuleLogical unit which weights hostsThe lower the betterWeights can be prioritized using Factors (defaults to 1)The result is a score table, which will be taken intoconsideration when scheduling the VM96

Weight ModulePredefined Weight ModulesEven DistributionEach host weight will be scored according to CPU loadPower SavingDefine Max Weightif (no VMs on Host) Max WeightElse (Max Weight – Even Distribution Weight)External Weight Modules written in python can beloaded into the engine97

Weight Module Example98

Weight Module Example99

Weight Module Example100

Load BalancingTriggers a scheduled task to determine which VMneeds to be migratedA single load balancing logic is allowed per clusterVMLoad balanceHost White ListMigrate VMSelected Hostschedule101

Load BalancingFor backward compatibility we have 2 predefined LoadBalancing algorithmsEven DistributionCalculates over-utilized and under-utilized hostsaccording to upper CPU load thresholdSelect a VM out of the over-utilized hostsPass VM and under-utilized hosts to the schedulermigrate VM to the host selected by the schedulerPower SavingSame as Even Distribution, but with a second thresholdfor low CPU load102

Load Balancing Example same as previous103

Load Balancing Example same as previous104

Load Balancing Example same as previous105

External Policy UnitsScanning directory /usr/share/ovirt-scheduler-proxy/plugins for pythonsource filesAnalyze for filter / weight / balance functionsCache resultsExpose source files as external policy units106

Cluster Policy Management107

Apply Cluster Policy108

VDSM Hooks Hooks CatalogueCommand Line Interface109

Hooks“Hook” mechanism for customizationAllows administrator to define

filter True for user-level API (default is False – admin API) persistent_auth True for using API REST sessions (default is False) . UI Plugin API (Demo) Scheduling API VDSM hooks. 51 UI Plugins Command Line Interface . 52 Web Admin user interface Extend oVirt Web Admin user interface. 53 Web Admin user interface. 54 Web Admin user interface . 55 Web Admin user interface. 56 Web Admin user .