Friday, 2016-02-26

*** yangyape_ has quit IRC00:02
*** dschroeder has quit IRC00:31
*** daemontool has quit IRC00:47
*** ddieterly has joined #openstack-freezer00:54
*** ddieterl_ has joined #openstack-freezer00:58
*** ddieterly has quit IRC00:58
*** yangyapeng has joined #openstack-freezer01:13
*** ddieterly has joined #openstack-freezer01:16
*** EinstCrazy has joined #openstack-freezer01:16
*** ddieterl_ has quit IRC01:19
*** yangyapeng has quit IRC01:39
*** EinstCrazy has quit IRC01:40
*** EinstCrazy has joined #openstack-freezer01:42
*** reldan has quit IRC01:45
*** ddieterly has quit IRC03:30
jith_hi all, when i try freezer-scheduler --help command to know its usage, i received the following error "ImportError: No module named apscheduler.schedulers.blocking"04:51
*** ddieterly has joined #openstack-freezer05:01
*** ddieterly has quit IRC05:05
*** ddieterly has joined #openstack-freezer06:02
*** ddieterly has quit IRC06:07
*** ddieterly has joined #openstack-freezer07:03
*** yangyapeng has joined #openstack-freezer07:04
*** ddieterly has quit IRC07:08
EinstCrazyhow did you install the freezer?07:10
jith_EinstCrazy: python setup.py install07:11
EinstCrazyIt seems the apscheduler in the requirement not install or not install correctly07:11
*** daemontool has joined #openstack-freezer07:31
*** hparekh has quit IRC07:55
*** zhangjn has quit IRC08:02
*** ddieterly has joined #openstack-freezer08:04
*** zhangjn has joined #openstack-freezer08:07
*** ddieterly has quit IRC08:09
*** daemontool has quit IRC08:35
*** daemontool has joined #openstack-freezer08:37
*** reldan has joined #openstack-freezer08:47
*** ddieterly has joined #openstack-freezer09:05
*** ddieterly has quit IRC09:11
jith_EinstCrazy: thanks.. sorry i was away09:23
jith_in command line, we will use only freezerc??? then whats the use of agent and scheduler?? .. couldnt find any documentation regarding that.. sorry if i m wrong09:33
daemontooljith_,  freezerc == freezer-agent09:39
daemontoolit's the same09:39
daemontoolfreezerc is the old name, freezer-agent is the new name09:39
daemontoolwe keep both for compatibility09:39
jith_daemontool: thanks.. i got09:40
jith_daemontool: incremental backup is available for volumes and instances???09:42
*** yangyape_ has joined #openstack-freezer09:42
daemontooljith_, this review deliver incremental backup for volumes https://review.openstack.org/#/c/276685/09:44
daemontoolinstances we still do not support that09:44
daemontoolthe incrementals I mean09:45
*** yangyapeng has quit IRC09:45
*** yangyape_ has quit IRC09:47
jith_thanks :)09:47
*** reldan has quit IRC09:48
jith_daemontool: so as of now, it wil take the whole instance backup??? in scheduler, during the second backup, it will delete the previous one and take a new backup??09:51
*** EinstCrazy has quit IRC09:55
daemontoolit won't delete the old backup10:06
daemontoolcan't remember the details10:06
daemontoolreldan implemented it10:07
daemontoolsorry10:07
daemontoolI can take a look at let you know asap tho10:07
*** ddieterly has joined #openstack-freezer10:07
jith_daemontool: thanks, its ok10:12
jith_if I use freezer for DR, I should be able to keep backup of nova instance and cinder volumes... is it? If so I should be able to take backup from outside of VM.. so scheduler is used for that purpose???10:12
jith_sorry for the trouble... :(10:12
*** ddieterly has quit IRC10:12
frescofjith_, the scheduler is needed if you want to use the api10:15
frescofbut it will run on the compute nodes in that case10:15
*** daemontool has quit IRC10:16
jith_frescof: thanks..10:19
jith_yes10:19
jith_frescof: u meant freezer api right??10:19
*** reldan has joined #openstack-freezer10:24
frescofye10:38
frescofs/ye/yes/10:38
jith_frescof: thanks10:53
*** jith_ has quit IRC10:54
*** ddieterly has joined #openstack-freezer11:08
*** ddieterly has quit IRC11:12
openstackgerritEldar Nugaev proposed openstack/freezer: Removing defaults values from cli.  https://review.openstack.org/28463711:13
*** EinstCrazy has joined #openstack-freezer11:32
*** reldan has quit IRC11:46
*** reldan has joined #openstack-freezer12:04
*** ddieterly has joined #openstack-freezer12:09
*** ddieterly has quit IRC12:13
*** daemontool has joined #openstack-freezer12:14
*** ddieterly has joined #openstack-freezer13:09
*** ddieterly has quit IRC13:14
*** daemontool has quit IRC13:15
*** daemontool has joined #openstack-freezer13:16
*** daemontool has quit IRC13:17
*** daemontool has joined #openstack-freezer13:18
*** daemontool has quit IRC13:24
*** reldan has quit IRC13:49
openstackgerritFausto Marzi proposed openstack/freezer-api: Switch to testr from pytest  https://review.openstack.org/26095013:54
*** reldan has joined #openstack-freezer13:54
*** daemontool has joined #openstack-freezer13:54
daemontoolreldan, ping13:56
reldandaemontool: yes13:56
daemontoolI've added few more requirements here https://etherpad.openstack.org/p/freezer_meetings13:57
daemontoolunder other requirements13:57
reldandaemontool: Great, looks good.13:59
reldandaemontool: I only don’t see requirements for tenant13:59
daemontoolI think any mention to vm instances and volumes14:00
daemontoolcan be considered probably related to tenant14:00
reldandaemontool: I can see only    Backup on the instance level, volumelevel, Relational/Non-Relational database and BigData Cluster.14:01
m3m0daemontool agentless backups?14:01
reldanbut it is about instance level, volume level, not tenant level14:01
daemontoolm3m0,  well that's what they gave me14:01
reldanI see requirements from you and from ddieterly and don’t see any item related to tenant14:01
daemontool:)14:01
daemontoolreldan,  I think any tenant specific resources14:02
m3m0I update the doc to have data consistency checks14:02
daemontoollike volumes and instances14:02
daemontoolis tenant14:02
m3m0who gave it to you?14:02
reldaneverything is good, but seems nobody ask about tenant14:02
reldanES backup (logs and Freezer jobs)14:02
reldanMetering (esp Billing related ones)14:02
reldanStorage Quota management ( I am not sure how much we have here)14:02
reldanVertica DB14:02
reldanSwift data backup14:02
reldanContainer backup14:02
daemontoola friend architect in Ercisson14:02
reldanes backup - not tenant, additional mode like mysql or mongo14:03
reldanmetering - don’t related to tenant14:03
daemontoolagree14:03
reldanstorage quota - great, but don’t related to tenant14:03
reldanvertica db - like mysql, mongo, es - don’t related to tenant14:03
reldanswift data backup - cool, but don’t related to tenant14:03
daemontoolI think tenant is: any metadata (i.e. from keystone), volumes, instances and objects14:03
reldancontainer backup - suppose the same as swift bacup14:03
reldandaemontool: Yes!14:04
reldandaemontool: But I don’t see any item related to metadata - networks, volumes, vm-s, floating ip-s14:04
daemontoolreldan,  in the ones I've added there's14:04
daemontool        Backup one instance daily and save for N days.14:04
daemontool    ·        Backup another instance every M minutesand save backups for D days.14:04
daemontoolCopy your snapshots to other OpenStack(Swift)14:05
reldanGood, but it is instance backup, not tenant14:05
daemontool    Recover instances and volumes in otherregions with a few simple mouse clicks.14:05
daemontoolwell14:05
daemontoolan instance is a tenant specific resource14:05
reldanYes, it is14:05
daemontoolin the strict sense you mean14:05
daemontoolprobably only the metadata is tenant only14:06
daemontoolbut I think we should backup and restore any tenant resources14:06
daemontoolso you are right14:06
daemontoolnetwork, keystone, volumes, isntances,14:06
daemontoolfloating ips14:06
daemontoolyes14:06
daemontoolthat14:06
daemontoolI agree14:06
reldanYes, but we need someone who needs this14:06
reldanwho has real requirements for transition let’s say real applicatoin14:07
reldanfrom one OS installation to another14:07
daemontoolyes14:07
reldanBut I don’t see requirements for this. We have a lot of requirements and fields of improvement, but I don’t see requirements for such transition14:07
*** zhangjn has quit IRC14:08
reldanLike  Backupscripts support for Linux & Windows servers – give your own hooks to backup applications consistently: MySQL, Oracle, MongoDB, XFS etc...14:08
reldanI understand what we should do this14:08
reldanAnd understand why it is important14:08
reldanWe can define it and start to work on it14:09
reldanBut I don’t understand what we want from tenant backup14:09
*** zhangjn has joined #openstack-freezer14:09
daemontoolI think reldan  you are right14:09
daemontoolI intend it as you are14:09
daemontoolso far14:10
reldandaemontool: Yes. I see a lot of requirements and we create many blueprints. But all hpe requirements are not tenant14:10
*** ddieterly has joined #openstack-freezer14:10
reldanIt is about new modes14:10
daemontoolwhat everybody wants is volumes and instances incremental backups from one OS deployment and restore to a different one14:10
reldanbilling and metering integration14:10
daemontoolthat is the basic we can offer14:10
daemontoolbilling I think we can write the integratino with cloudkitty14:11
daemontoolmetering I have no idea14:11
reldanYes, I can say. Tenant backup from our point of view - is creation of backup of one instance with attached volumes and relocation it to different OS cluster, without restore of network, sequrity rules and ip14:12
reldanAnd if we agree with this difinition - great, let’s go that way.14:13
reldanWe will define metadata for this task14:13
*** EinstCrazy has quit IRC14:14
*** ddieterly has quit IRC14:15
*** daemontool has quit IRC14:16
*** daemontool has joined #openstack-freezer14:16
openstackgerritFausto Marzi proposed openstack/freezer-api: Switch to testr from pytest  https://review.openstack.org/26095014:16
daemontoolreldan sorry got disconnected14:17
reldandaemontool: no worries.14:18
reldanreldan: Yes, I can say. Tenant backup from our point of view - is creation of backup of one instance with attached volumes and relocation it to different OS cluster, without restore of network, sequrity rules and ip14:18
reldan[2:13pm] reldan: And if we agree with this difinition - great, let’s go that way.14:18
reldan[2:13pm] reldan: We will define metadata for this task14:18
reldan[2:14pm]14:18
*** daemontool_ has joined #openstack-freezer14:18
reldandaemontool: I would prefer to check requirements list and create blueprints14:19
reldanthen discuss it14:19
reldanI see that anyone wants different application hooks and modes - like vertica, elastic search, give your own hooks to backup applications consistently: MySQL, Oracle, MongoDB, XFS etc...14:20
*** daemontool has quit IRC14:22
*** yangyapeng has joined #openstack-freezer14:26
m3m0reldan daemontool should the python-freezerclient be able to restore a backup?14:29
m3m0daemontool any update on the python-freezerclient repo?14:30
m3m0this is WIP: https://github.com/memogarcia/python-freezerclient/tree/cliff14:30
openstackgerritFausto Marzi proposed openstack/freezer-api: Switch to testr from pytest  https://review.openstack.org/26095014:36
daemontool_m3m0,  let me finish the testr thing in the freezer-api and we'll work on that14:39
m3m0kkk14:41
*** ddieterly has joined #openstack-freezer14:49
*** ddieterly has quit IRC14:59
*** daemontool_ has quit IRC15:01
*** daemontool has joined #openstack-freezer15:05
*** ddieterl_ has joined #openstack-freezer15:08
ddieterl_daemontool: it looks like there currently are no integration tests for freezer-api, only unit tests15:09
ddieterl_is that correct?15:09
*** ddieterl_ is now known as ddieterly_15:09
daemontoolddieterly_,  I think you are correct15:12
daemontoolin theory the integration tests for the api, are the integration tests of the scheduler15:13
daemontoolbut integration tests like tempests15:13
daemontoolnope, we do not have it15:13
daemontoolat all15:13
ddieterly_yea, ok, we need to get the unit tests to run in the gate for the freezer-api, correct?15:14
ddieterly_right now, it looks like the unit tests are not running in the gate15:14
daemontoolddieterly_,  exactly15:15
daemontoolthat's why I'm trying to do this https://review.openstack.org/#/c/260950/15:15
daemontoolbefore15:15
daemontooltoday should be ok to pass all the verifications15:15
ddieterly_does that actaully make the tests run in the gate?15:16
daemontoolddieterly_, nope15:17
ddieterly_i thought that there was some config in openstack-infra that needs to be done to get the tests to run15:17
daemontoolbut it solve the current issues15:17
ddieterly_ok, that's what i thought15:17
daemontoolthat are blockers to do that15:17
ddieterly_there aren't any blockers15:17
daemontoolthe dsvm gate job fails15:17
ddieterly_just need to config openstack-infra, i believe15:17
daemontooltestr is not being used15:18
daemontoolthe test-requirements15:18
daemontoolare not part of the global-requirements15:18
daemontoolso the verification fails15:18
daemontoolthere are multiple blockers :)15:18
ddieterly_after you patch to use testr, then it shold work if we get it to run in the gates, yes?15:18
daemontoolyes15:18
daemontoolI mean the verification is ok15:19
daemontoolbut you are right15:19
daemontoolthe integration tests are not15:19
ddieterly_so, is the problem that your patch is failing the gates?15:19
daemontoolno, the problem is that master currently for the freezer-api is failing at the gates15:19
daemontoolthat change should fix it15:19
ddieterly_https://review.openstack.org/#/c/260950/ should fix the gates?15:20
daemontooli.e. porting the freezer-api code to py3415:20
daemontoolyes15:20
ddieterly_ok, so why is your patch not being +2?15:20
daemontoolbecause it still doesn't verify15:20
daemontoolas I'm working on it15:20
ddieterly_ok, why does it not verify?15:21
ddieterly_i don't see check jobs in the patch at the moment15:21
daemontoolddieterly_,  http://status.openstack.org/zuul/ search for freezer-api15:21
ddieterly_ok, so this is the failure? http://logs.openstack.org/50/260950/8/check/gate-freezer-devstack-dsvm/47df3c2/15:22
daemontoolthat was one of the failures yes15:22
daemontoolbefore there was pep815:22
daemontoolbefore pylint15:22
daemontoolbefore py3415:22
ddieterly_forgive me if i'm asking dumb questions15:22
daemontoolnot a prob at all15:22
daemontoolcause15:23
daemontoolafter we enabled15:23
daemontoolthe python-jobs15:23
daemontoolon openstack-infra15:23
daemontooland the check requirements jobs15:23
ddieterly_is that gate that is failing automatically running the unit tests at this point or do we need to configure something in openstack-infra to get the unit tests to run?15:23
daemontool(which are necessary to be added in the project list in openstack/requirements and to add new deps in global-requirements)15:23
daemontoolddieterly_,  I think we can fix the gate without modifying anything15:24
daemontoolin project-config15:24
ddieterly_ok, so nothing needs to change in openstack-infra (you think)?15:24
daemontoolI think we do not have to change  anything in infra15:24
daemontoolto solve our curernt issues15:24
daemontoolare matters of path15:24
ddieterly_ok, do you need help with figuring out why that patch is failing?15:25
daemontoolI think I did today15:25
daemontoolwhat would be good15:25
daemontoolis to automatically15:25
daemontoolexecute the integration tests15:25
daemontoolin the freezer agent15:25
daemontoolagainst the api, with the dsvm gate job15:25
daemontoolbut that would be after this15:26
daemontoolcause if we do not enable tests to be executed automatically15:26
daemontoolthis is crap15:26
ddieterly_yea, we need to get this patch working15:26
daemontoolok :)15:26
ddieterly_agreed about the crap ;-)15:26
daemontool:)15:27
ddieterly_if we don't have integration tests, then things will break when people modify code15:27
daemontoolyes15:27
ddieterly_ok, so you have this patch under control?15:27
daemontoolI think so15:27
daemontoollet's see hot it goes15:27
daemontools/hot/how/15:27
ddieterly_ok15:27
daemontoolthe current verification15:28
daemontoolI think something needs to be backported also to liberty15:28
daemontoolon freezer-api after that15:28
ddieterly_ok, then if that patch goes ok and is merged, then we can try to get some integration tests on the freezer-api15:28
ddieterly_brb15:29
daemontoolty15:29
ddieterly_to be all openstack-like, we would need tempest tests on the freezer-api, right?15:34
ddieterly_daemontool: ^^15:34
ddieterly_(not that i like writing tempest tests)15:35
daemontoolddieterly_,  yes15:35
ddieterly_ok, i've been doing monasca, and we skirted a lot of openstack conventions, so i need to come up to speed on the official openstack way15:35
ddieterly_so, a good place for me to start contributing would be tempest tests on freezer-api?15:36
daemontoolyes15:36
ddieterly_alright15:36
daemontoolI think15:37
daemontoolis we can define like 10 scenarios of real world backups15:37
ddieterly_do you know what the mechanism is for the unit test to get run in the gate? where is that confgured in openstack-infra?15:37
ddieterly_i looked last night, but i could not figure out how the unit tests get started in the gate15:37
daemontoolddieterly_,  do you mean from the dsvm job_15:39
daemontool?15:39
daemontoolor just15:39
daemontoolthe py27 tests?15:39
daemontoolfrom tox.ini15:39
ddieterly_in the dsvm gate job, the py27 tests get run, don't they?15:39
daemontoolnope15:39
daemontoolpy27 has it's own job15:39
daemontoolpart of the python-jobs jobs group15:40
daemontoolpy27, py34, docs15:40
daemontoolpep815:40
ddieterly_ok, in this job http://logs.openstack.org/50/260950/8/check/gate-freezer-api-python27/f7f28e4/ how do the tests get started?15:40
ddieterly_what in openstack-infra runs that gate and starts the tests?15:40
daemontoolwith tox15:40
daemontoolexecute tox15:41
daemontoolwith the content of tox.ini15:41
daemontoolhttps://review.openstack.org/#/c/260950/11/tox.ini15:41
ddieterly_where is that configured? is it explicity configured or is it just implicitly done?15:41
daemontoolimplicitly15:41
ddieterly_so, if there is a tox.ini file, then openstack-infra will automatically run the unit tests?15:42
ddieterly_again, sorry if this is a dumb question15:42
daemontoolif there's setup.py15:43
daemontooland tox.ini file15:43
ddieterly_is this all documented somewhere?15:43
daemontooland the jobs specified15:43
daemontoolin the templates15:43
daemontoolyes one sec15:43
daemontoolin the project creators guide15:43
daemontoolhttp://docs.openstack.org/infra/manual/creators.html15:43
daemontoolunder "Configure Zuul to Run Jobs"15:44
ddieterly_thank you!15:44
daemontoolyw15:44
ddieterly_great, i'll read this instead of bombarding you with questions15:44
ddieterly_then, i'll start looking into getting integration tests running15:45
ddieterly_about mid-cycle15:46
ddieterly_it would be better if we just meet at summit and i skip the mid-cycle15:46
ddieterly_mid-cycle is too close to the summit15:47
ddieterly_we can then plan to have another mid-cycle after the summit15:47
daemontoolyes15:47
daemontoolsure15:48
daemontoolddieterly_,  let's talk with the others15:48
daemontoolwe should see who can go to the summit and who can't15:48
ddieterly_ok, i did not think of that15:48
ddieterly_would be good to meet everyone15:49
ddieterly_but travelling to galway and then travelling to austin is a bit much for me15:49
daemontoolddieterly_,  and for the record15:49
daemontoolbefore I was mistaken15:49
daemontoolto fix the dsvm error15:49
*** ddieterly_ is now known as ddieterly15:49
daemontoolwe need to modify something in the project-config repo in openstack-infra15:49
ddieterlyyea, i thought that we needed to do something there15:50
daemontoolprecisely the jenkins/jobs/freezer.yaml15:50
ddieterlybash: line 1: ./post_test_hook.sh: No such file or directory15:53
daemontoolddieterly,  https://review.openstack.org/#/c/285384/15:53
daemontoolyep15:53
daemontoolthat's the fix15:53
ddieterlyok15:54
openstackgerritFausto Marzi proposed openstack/freezer-api: Switch to testr from pytest  https://review.openstack.org/26095015:55
ddieterlycool, thanks, ttyl15:55
*** yangyapeng has quit IRC15:56
*** yangyapeng has joined #openstack-freezer15:56
*** yangyapeng has quit IRC15:56
daemontoolddieterly,  it took many weeks to understand the jobs and infra related stuff15:58
ddieterlyyes, i have wrestled with it before when i setup the gate for monasca-api15:59
ddieterlyi hate it15:59
daemontoolhigh maintenance16:00
daemontoolI'll be back in 20 min16:03
*** dschroeder has joined #openstack-freezer16:05
ddieterlyk16:05
*** daemontool has quit IRC16:07
*** daemontool has joined #openstack-freezer16:40
daemontoolreldan, there's an interesting idea frescof had for incremental backups of of vms16:49
daemontoolplease have a conversation with him16:49
daemontoolso we can write a bp16:49
reldandaemontool: good16:50
daemontoolwe need rsync code in tho16:51
daemontooldoh!16:51
daemontoollol16:51
daemontool:)16:51
daemontoolbut the approach is quite smart16:51
daemontoolas it leverage the same code base16:51
daemontool:)16:51
daemontooltalk with him16:51
reldandaemontool: yes, what is about rsync?16:51
reldandaemontool: are we going to intergrate it?16:51
daemontoolyes16:53
daemontoolas soon as I finish this workload related to the repos16:53
daemontoolthat we have to do16:53
daemontoolbecause we are an official openstack project now16:53
daemontoolI'll invest hard on that16:53
daemontoolthere's some code16:53
daemontoolI have been working locally16:54
daemontoolafter that commits were abandoned16:54
daemontoolI have to submit for review16:54
daemontoolalso16:54
daemontoolI'd like the freezer-agent16:54
daemontoolexecute with pypy16:54
daemontoolI did few tests months ago16:54
daemontooland it was really worth16:54
daemontoolmore and more for rsync16:54
daemontoolreldan,  after the first release16:58
daemontoolif you want we can work on the optimization16:58
daemontoolhow to make it better16:58
daemontooland so on16:58
daemontoolbut we need absolutely to have the following features16:59
daemontool1) volumes incrementals, restorable on a another os16:59
daemontool2) vms incrementals, restoreable on another os16:59
daemontool3) block based (preferibly rolling checksum)16:59
reldanvm - how about floating ip - we cannot restore floating ip on different os17:00
daemontool4) integration tests enhanced and executed from dsvm17:00
daemontoolreldan,  good point, we have to provide a solution for that17:00
daemontoollike reassign floating ips17:01
reldanbut I don’t think it is possible. you have an ip range per os installation17:01
reldanand cannot reassing ip from one os installation to another17:01
reldanby api17:01
daemontoolyou can have the same fqdn, same dns records and different ips17:02
daemontoolthat would be the limitation17:02
daemontoolyou can restore to a differnt os, same fqdn, same dns entried in designate17:02
daemontoolbut different floating ips17:02
reldanyes, agree but floating ip will be different.17:03
daemontooltotally agree17:03
daemontoolyes17:03
daemontoolthat wil be our limitation17:03
*** reldan has quit IRC17:05
*** szaher_ has joined #openstack-freezer17:19
*** szaher has quit IRC17:20
*** reldan has joined #openstack-freezer18:57
*** ddieterly has quit IRC19:37
*** reldan has quit IRC19:39
*** ddieterly has joined #openstack-freezer19:41
*** reldan has joined #openstack-freezer19:56
*** reldan has quit IRC20:07
*** reldan has joined #openstack-freezer20:13
*** reldan has quit IRC20:32
*** ddieterly has quit IRC21:39
*** ddieterly has joined #openstack-freezer21:40
*** daemontool_ has joined #openstack-freezer23:10
*** daemontool has quit IRC23:13
*** ddieterly has quit IRC23:35
*** yangyapeng has joined #openstack-freezer23:35

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!