Tuesday, 2016-03-01

*** EmilDi has quit IRC00:33
*** dschroeder has quit IRC00:39
*** yangyapeng has joined #openstack-freezer01:01
*** yangyapeng has quit IRC01:02
*** EinstCrazy has joined #openstack-freezer01:06
*** yangyapeng has joined #openstack-freezer01:10
*** jith_ has quit IRC02:32
*** jith_ has joined #openstack-freezer04:47
jith_Hi all, is freezer-api required in between  backingup in ssh location and  freezer-scheduler...04:51
jith_means freezer scheduler will work only with freezer-api??  because scheduler uploads job to api...04:53
*** dmellado has quit IRC06:34
*** dmellado has joined #openstack-freezer06:39
*** jith_ has quit IRC07:40
*** daemontool__ has joined #openstack-freezer08:38
*** daemontool_ has quit IRC08:41
*** openstackgerrit has quit IRC08:48
*** openstackgerrit has joined #openstack-freezer08:48
*** jith_ has joined #openstack-freezer09:02
openstackgerritFabrizio Vanni proposed openstack/freezer: enable output of metadata to a file  https://review.openstack.org/28462909:25
*** yangyapeng has quit IRC09:40
*** yangyapeng has joined #openstack-freezer09:40
*** EinstCrazy has quit IRC09:54
daemontool__jith_,  hi09:54
daemontool__regarding your questions09:54
*** EinstCrazy has joined #openstack-freezer09:54
daemontool__the freezer-api are not required to take backup and store them to an ssh node09:55
daemontool__the scheduler works also without api09:55
daemontool__there's an option if I remembe rwell called --no-api09:55
jith_daemontool_: thanks09:56
daemontool__jith_,  to take backups and store them to an ssh node09:59
daemontool__you do not even need the scheduler if you do not want it for some reason09:59
*** EinstCrazy has quit IRC10:15
*** EinstCrazy has joined #openstack-freezer10:16
*** EinstCrazy has quit IRC10:21
*** yangyapeng has quit IRC10:23
*** reldan has joined #openstack-freezer10:32
*** reldan has left #openstack-freezer10:36
*** daemontool has joined #openstack-freezer10:36
jith_daemontool_ :thanks.. sorry i was away10:39
*** daemontool__ has quit IRC10:40
jith_ok but i want to take daily backups.. so scheduler is needed right?10:40
*** reldan has joined #openstack-freezer10:42
vannif_the scheduler-api infrastructure provide some features to the backup service: a single endpoint to interact with to distribute and monitor the execution of the backups on different clients or nodes.10:43
vannif_if the problem is really small (single node with recurring backups) you could use a simple cron job to execute the freezer-agent.10:45
jith_so if i am enabling freezer in openstack, freezer-api and freezer should be in controller as well as compute nodes???10:45
jith_vannif: thanks i got10:45
vannif_you're welcome. what is your objective ? single node ? multiple VM ? backup of OpenStack control plane ?10:46
jith_backup of openstack control plane10:47
jith_as of now.. i am trying to execute in single node10:50
jith_scheduler should be in all nodes or only controller node?10:51
reldanPlease review https://review.openstack.org/#/c/284637/10:55
vannif_https://review.openstack.org/#/c/284629/11:02
m3m0vannif_ done11:08
m3m0reldan just a quick question in the patch11:08
reldanyes11:08
m3m0i wrote the question in the patch11:09
reldanm3m0: nope currently we have 2 places with defaults11:10
reldan1) cli 2) dictionary called DEFAULT_PARAMS11:10
reldanand it just has no sense11:10
reldanso if you don’t provide mode, mode should be taken from dict11:11
reldanm3m0:  'mode': 'fs'11:11
m3m0ok ok make sense11:14
reldanm3m0: Thank you!11:15
vannif_jith_: the scheduler is the process that contacts the api, fetches the jobs and passes the instructions to the freezer-agent to execute the backups (or whatever action is written in the job document). You need to run it on any node that needs to execute the freezer-agent11:16
openstackgerritMemo Garcia proposed openstack/freezer-web-ui: Improve success messages when it takes time to display the action  https://review.openstack.org/28650711:16
m3m0slashme, did we provide the test environment to the QA guys?11:17
jith_vannif_ thanks, i got that.. so scheduler should run in all nodes??11:18
daemontoolall, FYI https://review.openstack.org/#/c/271072/11:19
daemontooljith_,  the scheduler run on any node where the data you want to backup is11:20
daemontoolso yes, in theory on any node11:20
jith_so if i want to take a backup of a nova instance, i should upload the scheduler job from the particular compute node?11:20
daemontoolif you want to backup a nova instance, it is not required as currently we use the nova api11:21
daemontoolso the scheduler can run on the instance itself11:21
daemontoolthe concept is> if you have to backup files on the file system then the scheduler needs to run on the node where the data is11:21
daemontoolif you have to backup volumes and vms, then you can run the scheduler/agent where you want11:22
jith_fine.. cant i take backup of instance  from compute node??11:22
daemontoolyes11:22
daemontoolah ok11:22
daemontooldo you mean11:22
daemontooldirectly from the file system like /var/lib/nova/instances?11:22
daemontoolrather than using the nova api? something liek that?11:23
jith_daemontool: yes yes11:23
daemontooljith_, good question11:23
daemontoolwe are working on that11:23
daemontoolnot currenlty11:23
daemontoolI mean11:23
daemontoolyou can11:23
daemontoolthe problem is that if you backup instances from /var/lib/nova/instances11:23
daemontooland then restore11:23
daemontoolthe state of the vms can be inconsistent with the nova db in mysql11:24
daemontool:(11:24
daemontoolso we are working on delivering a feature that will solve that11:24
daemontoolreldan, is the man11:24
daemontoolfrescof, ^^11:24
vannif_regardin the creation of the the job: it doesn't need to be created on the same node. in fact the common usage is to have the scheduler running on the respective nodes, while the jobs are created, uploaded, managed from a managment point, for example a admin node or using the horizon dashboard which is under development as well11:25
jith_daemontool: ok fine... also i saw a line in the freezer doc11:25
jith_ Execute a nova backup:11:25
jith_$ freezerc --nova-inst-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b11:25
daemontooljith_,  so we need to provide a mechanism that backup the instance incrementally, save the related metadata11:25
daemontooland then restore it11:26
daemontoolyes11:26
vannif_yes. if you backup the ephemeral disk, it's not even granted that the VM can boot. you're backin up a possibly inconsistend filesystem11:26
jith_fine.. then for cinder volume??11:27
jith_Execute a cinder backup:11:27
jith_$ freezerc --cinder-vol-id 3ad7a62f-217a-48cd-a861-43ec0a04a78b11:27
daemontooljith_,  that just wrap this nova api http://developer.openstack.org/api-ref-compute-v2.1.html#createBackup11:28
daemontoolunfortunately it does not provide incrementals11:28
daemontoolatm11:28
jith_daemontool: oh fine.. thanks11:29
jith_cinder volume backup is incremental??11:29
daemontooljith_,  yes11:30
daemontoolbut it does the same11:30
daemontoolit uses the cinder incremntals api11:30
daemontooland that is from liberty11:30
jith_backup service?11:30
daemontoolcurrently the limitation is11:31
jith_cinder backup service?11:31
daemontoolyes11:31
daemontoolthe limitation is that you can't backup volumes on an openstack instance and restore them on an independent openstack isntance11:31
daemontoolcause we need to backup also the metadata to do that11:31
jith_ok thanks for responding to my questions patiently11:31
daemontoolalso currently the backups are not removed using cinder backups11:32
jith_yes11:32
daemontoolI mean removed automatically11:32
daemontoolthat should be also delivered in Mitaka11:32
jith_ok i got11:32
daemontoolreldan, any thoughts/says on this?11:32
daemontool:)11:32
reldandaemontool: I actually don’t know what to say. Currently nova/cinder backup in freezer - it is just a calls of api11:33
reldandaemontool: nothing actually more11:33
daemontoolok11:33
daemontoolreldan,  when you can, please have a conversation with frescof11:34
daemontoolhe found an easy way11:34
daemontoolto execute incrementals backup11:34
daemontoolfor vms11:34
daemontool(it requires rsync tho)11:34
daemontoolbut until that conversation&bp happens11:34
daemontoolwe can probably keep working on improving the volumes backups?11:34
daemontoollike adding metadata backup11:35
daemontooland managing backup removal/retention?11:35
reldandaemontool: Yes, I would like to have something like architecture. What we are going to have for cinder/nova etc.11:35
reldandaemontool: And what we are not going to support11:35
reldandaemontool: Like would we want to support aws in feature?11:36
reldandaemontool: Or only openstack nova/volume11:36
reldandaemontool: or vmware11:36
daemontoolok11:36
reldandaemontool: If we want to support different cloud providers - we should think about architecture. How to isolate layers11:37
daemontoolreldan,  I agree11:37
daemontoolthe thing is that we do not have any use case for that...11:37
daemontoolit's a great idea11:38
reldandaemontool: if we have no plans to support other cloud providers - we can just write cinder specific code and it is all11:38
daemontoolwe can do it11:38
daemontoolbut no one asked for it11:38
daemontoolwe can think abou tit11:38
daemontoolwhat we really need is to provide users to manage volumes and vm instances incremental backups11:39
reldanabout incremental nova - it would be great to read somewhere about this. like a blueprint or just several sentences in etherpad should be enough11:39
daemontoolwith automatic deletion/retention11:39
daemontoolfrescof,  is writing that11:39
reldangreat!11:39
daemontoolas soon as he finish to add the info to the bp11:39
daemontoolhe should have written the bp11:39
daemontoolyesterday11:39
daemontoolfor dr11:39
daemontoolonce we provide that features...11:40
daemontoolI think we can make better as much as we want11:40
daemontoolbut let's think abou tit11:40
*** reldan has quit IRC11:45
jith_freezer-api is needed in all nodes??.. i saw in architecture, API runs in controller node... but how come scheduler know API's credential...  the below command should run in controller ???11:45
jith_ upload that job into the API:11:45
jith_freezer-scheduler -c node12 job-create --file test_job.json11:45
jith_-c --> will be client node where backup files reside11:46
jith_i am sorry if i m confusing...11:46
daemontooljith_,  that are the normal credentials you have with OS_* env variables11:47
daemontoolonce the freezer-api are registered to keystone11:47
daemontoolthe api endpoint will be available11:47
daemontoolfrom the keystone catalog11:47
daemontoolto the scheduler authenticate to keystone (using the OS_* env var)11:48
daemontools/to/so/11:48
jith_oh yes i forgot that..11:48
daemontoolthen retrieve the freezer-api endpoint11:48
daemontoolok11:48
daemontool:)11:48
daemontoollol11:48
jith_client_id is?? its own hostname??11:49
daemontoolit should be tenant_id + hostname11:53
daemontoolvannif_,  does the client_id is generated with OS_PROJECT_ID if OS_TENANT_ID is not found?11:55
daemontooljith_,  do you mind add all these information you asked to the openstack/freezer/README  in the related sectino?11:56
daemontool:)11:56
jith_daemontool: thanks: we should specify  the parameter right while executing scheduler???11:59
jith_never.. :)..but how can i add??12:01
daemontooljith_, clone the freezer repo12:01
daemontooledit the README file12:02
daemontoolmake the docs better by providing these information12:02
daemontooland send it for review12:02
jith_ sure..12:02
daemontooldo you know how to use git?12:02
jith_oh fine.. ya little bit12:02
daemontoolgit clone https://github.com/openstack/freezer12:02
daemontoolcd freezer12:02
jith_ok12:02
daemontoolgedit README.rst12:03
daemontooledit12:03
daemontoolgit add README.rst12:03
jith_will edit README file.. then how to send it??12:03
daemontoolgit commit12:03
jith_oh ok12:03
daemontooladd to the commit message a description12:04
daemontoolalso add a bug previously opened on https://bugs.launchpad.net/freezer12:04
daemontoolthen12:04
daemontoolgit review -v12:04
daemontooland wait someone from us will review it12:04
daemontoolso you have your first contribution in openstack :)12:04
jith_done..12:05
*** EmilDi has joined #openstack-freezer12:05
*** reldan has joined #openstack-freezer12:05
jith_but couldnt get.. also add a bug previously opened on ""12:05
jith_didnt get exactly12:05
jith_sure :).. thanks for the great idea12:06
jith_i didnt get this..  also add a bug previously opened on https://bugs.launchpad.net/freezer...12:07
*** daemontool_ has joined #openstack-freezer12:07
daemontool_jith_, clone the repo12:07
daemontool_and add to the readme the information12:07
daemontool_that's the most important part12:07
daemontool_:)12:07
daemontool_after that is done12:08
daemontool_let me know12:08
daemontool_and we'll move forward12:08
daemontool_:)12:08
*** daemontool has quit IRC12:10
jith_daemontool_ : thanks.. sure sure12:13
daemontool_:)12:18
*** daemontool__ has joined #openstack-freezer12:34
*** daemontool_ has quit IRC12:37
*** EmilDi has quit IRC12:58
reldanszaher , m3m0 , daemontool_ , vannif_ , slashme : Guys please tell me what we should do with this https://review.openstack.org/#/c/280602/ ? I can abandon it or should we accept it? Because from my point of view, this change is able to make our life much easier.13:22
daemontool__I think we should to forward with that13:29
daemontool__s/to/go/13:29
reldandaemontool__: So, I’m going to work with comments then.13:34
daemontool__reldan,  yes please13:34
m3m0let's move forward with this13:36
daemontool__m3m0,  ty13:37
slashme+113:37
reldanThank you13:37
reldanFix for metadata curr_level https://review.openstack.org/#/c/286592/13:37
reldanSeems that our robot is dead13:37
daemontool__m3m0,  I think most of your comments make sense https://review.openstack.org/#/c/280602/14:02
daemontool__reldan,  do you agree/disagree with comments from m3m0  szaher  there?14:02
openstackgerritJin Nan Zhang proposed openstack/freezer-web-ui: Fixed importError urlresolvers  https://review.openstack.org/28609414:03
reldandaemontool_: do you like name mod for this? I have no strong opinion, if everyone agree to go with mods - let’s go14:04
*** EmilDi has joined #openstack-freezer14:04
reldandaemontool_: Here https://review.openstack.org/#/c/280602/4/freezer/hook/hook.py everything is good, but I suppose we should add it later14:05
reldandaemontool_: https://review.openstack.org/#/c/280602/4/freezer/hook/mongo.py yes, we should - but I suppose it should be done later. The idea of this pull request is structure refactoring.14:05
reldanhttps://review.openstack.org/#/c/280602/4/freezer/hook/mysql.py - the same14:06
reldanhttps://review.openstack.org/#/c/280602/4/freezer/job.py - really good idea, but should think about it a little bit more. Suppose it should be next step14:07
reldanhttps://review.openstack.org/#/c/280602/4/freezer/lvm.py - great idea, next step14:07
zhangjnwho can review some patch for freezer-web-ui?14:07
reldandaemontool__: So let me change hook to mod and add todo’s with good comments from szaher and m3m014:08
daemontool__m3m0,  is this good to go? https://review.openstack.org/#/c/278366/114:10
daemontool__did slashme  tested in your env?14:10
zhangjnHow to test the patch?14:13
zhangjnI can test it14:14
zhangjn https://review.openstack.org/#/c/278366/14:15
daemontool__m3m0,  ^^14:17
daemontool__slashme,  ^^14:17
daemontool__you have to enable TLS on the freezer-api14:18
daemontool__then test if the web ui works14:18
zhangjnthanks, I will try it.14:19
*** EmilDi has quit IRC14:28
m3m0TLS support in freezer-api is done with the config file, so no need to modify the code daemontool14:31
m3m0yes it is good to go14:32
m3m0zhangjn: which commit do you want review?14:33
zhangjnhttps://review.openstack.org/#/c/286094/14:36
m3m0there is one issue now, we should work with the stable/liberty version of horizon14:36
daemontool__m3m0, so do we need to merge this code? https://review.openstack.org/#/c/278366/14:37
m3m0could you try freezer-web-ui with that version? as soon as we move to mitaka we should be able to merge this patch14:37
zhangjnThis code looks good.14:37
zhangjnbut not to test it.14:38
m3m0agree with you, we need to start working on that14:38
m3m0as soon as I finish the python-freezerclient I'll sync up with the ui for the testing14:39
m3m0would you like to participate?14:39
m3m0yes daemontool14:39
m3m0good to go14:39
daemontool__ok14:39
*** EinstCrazy has joined #openstack-freezer14:39
m3m0zhangjn: do you know what's the best way to deploy javascrit code in horizon?14:40
zhangjnnow stable/liberty freezer-web-ui is available?14:40
m3m0yes an stable/liberty is available for freezer-web-ui14:42
zhangjnmaster is  unavailable now?14:42
zhangjnok, I will change the branch to stable/liberty to learn.14:43
m3m0horizon stable/liberty -> freezer-web-ui stable/libery and master,    horizon master -> we need to start working on that and your patch will help a lot14:43
m3m0I can give you a code walktrhough if you like14:43
zhangjnthanks, I will speed some time to learn it and ask for you.14:46
m3m0feel free to ask me anything14:47
m3m0my email is guillermo.ramirez-garcia@hpe.com14:47
daemontool__m3m0,  are you available to give the code walkthrought next thursday?14:47
daemontool__like we did last thursday14:47
daemontool__this week vannif_  could give the it for the scheduler14:47
daemontool__vannif_,  ^^14:47
m3m0this week I can't I was planning everything for the 4th week14:48
daemontool__yes14:49
daemontool__this week we can do the scheduler14:49
*** EinstCrazy has quit IRC14:50
*** EinstCrazy has joined #openstack-freezer14:50
zhangjnm3m0: is friendly to me. LOL14:52
*** daemontool_ has joined #openstack-freezer14:54
*** daemontool__ has quit IRC14:56
daemontool_ok flight booked, I'll be in Galway from 14-17 of March :)15:05
slashmedaemontool_: Nice15:11
*** EinstCrazy has quit IRC15:12
m3m0nice :)15:17
reldandaemontool_: Give me please +workflow https://review.openstack.org/#/c/284637/15:24
*** daemontool__ has joined #openstack-freezer15:24
reldandaemontool__: Give me please +workflow https://review.openstack.org/#/c/284637/15:24
reldanGuys give me +workflow please  https://review.openstack.org/#/c/284637/15:27
*** daemontool_ has quit IRC15:28
dmelladoreldan: isn't the patch name quite misleading?15:44
reldandmellado: Probably you are right15:44
reldandmellado: But we have very strange approach for receiving arguments15:45
openstackgerritMerged openstack/freezer-web-ui: fixed compilemessages error when install use devstack  https://review.openstack.org/28363115:45
dmelladowhat would be the usecase after the patch? I saw m3m0 asking in the PS but didn't see any reply15:45
*** dschroeder has joined #openstack-freezer15:46
reldandmellado: everything should work perfectly. Currently we have a dict called DEFAULT_PARAMS and default params in cli arguments.15:47
m3m0ok guys, this is the temp repo for python-freezerclient https://github.com/memogarcia/python-freezerclient/commits/master15:47
reldanand we can just remove duplicated default args from cli, because they just have no sense15:48
dmelladoreldan: how about chaning the commit message to somthing like 'Move default params to dict' instead?15:48
dmellados/chaning/changing15:48
reldanfor example for confg file - we are still have defaults, but from dict15:48
reldandmellado: They already there15:48
reldanI didn’t move, just delted it15:48
* dmellado rechecking15:48
m3m0szaher https://github.com/memogarcia/python-freezerclient/commits/master15:49
openstackgerritMerged openstack/freezer: Removing defaults values from cli.  https://review.openstack.org/28463715:49
dmelladoreldan: yep, then 'Change **** values' ;)15:49
dmelladoanyway xD15:50
dmelladogot merged, but It'd be great to get a more accurate commit message for next times, otherwise it's hard to get the meaning15:50
reldandmellad: Agree, but our code just sometimes have no meaning :)15:51
reldandmellado: As far as I understand. If we have command like that freezerc —config /tmp/config —storage=local —container=/tmp/backup15:52
reldandmellado: we want to read config and then overright storage and container15:52
dmelladooverride?15:53
dmelladocool15:53
reldandmellado: Yes. But here is a problem. We have default values. And let’s say I have —storage=local in config and my command is  freezerc —config /tmp/config —storage=swift15:54
reldandmellado:15:54
reldancli_options = dict([(x, y) for x, y in CONF.iteritems() if y !=15:54
reldan                        defaults.get(x) and y is not 'False'])15:54
dmelladoI see15:55
reldanSo here, I would like to override not default value from config with default value swift15:55
reldanand that code will check that swift is default and don’t override it15:55
reldancli_options = dict([(x, y) for x, y in CONF.iteritems() if y is not None])15:55
reldanThis code should work, because I have None for every default parameter from cli15:56
reldanand there is no possiblity to provide None as argument, that I may be sure - if I have None, user didn’t provide any value15:56
dmelladoreldan: got it, thanks for the explanation ;)16:00
reldandmellado: Thank you. I will try to have better description for my commits :)16:01
dmelladothanks! ;)16:01
m3m0daemontool__ can you create the pypi project for python-freezerclient ?16:01
openstackgerritFabrizio Vanni proposed openstack/freezer: lvm snapshot unique name  https://review.openstack.org/28670516:43
openstackgerritEldar Nugaev proposed openstack/freezer: Refactoring of mysql, msserver and mongo code.  https://review.openstack.org/28060216:46
openstackgerritFabrizio Vanni proposed openstack/freezer: lvm snapshot unique name  https://review.openstack.org/28670516:53
m3m0.17:08
*** reldan has quit IRC18:06
*** openstackgerrit has quit IRC18:18
*** openstackgerrit has joined #openstack-freezer18:18
daemontool__m3m0,  yes, but does it works?18:42
daemontool__I mean, if we remove the apiclient and use the python-freezerclient as dependancy in the web ui and scheduler18:43
daemontool__does it works?18:43
*** dmellado has quit IRC18:48
*** openstack has joined #openstack-freezer19:17
m3m0yes :) but not all the functionality is there yet19:20
*** openstackgerrit has joined #openstack-freezer19:20
m3m0daemontool__19:20
daemontool_m3m0,  ok19:21
*** smekel_ has quit IRC19:25
*** smekel_ has joined #openstack-freezer19:31
*** daemontool__ has joined #openstack-freezer20:05
*** daemontool_ has quit IRC20:09
*** reldan has joined #openstack-freezer20:31

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!