*** dschroeder has quit IRC | 00:19 | |
openstackgerrit | Deklan Dieterly proposed openstack/freezer: Add check for name=='__main__' https://review.openstack.org/266552 | 01:07 |
---|---|---|
*** yangyapeng has joined #openstack-freezer | 02:17 | |
yangyapeng | Hello | 02:17 |
yangyapeng | I have a few new patch,please review it. | 02:18 |
yangyapeng | DuncanT: hi | 02:19 |
*** EinstCrazy has joined #openstack-freezer | 03:03 | |
*** samuelBartel has joined #openstack-freezer | 08:29 | |
*** daemontool has joined #openstack-freezer | 09:20 | |
*** openstackgerrit has quit IRC | 10:02 | |
*** openstackgerrit has joined #openstack-freezer | 10:03 | |
*** daemontool_ has joined #openstack-freezer | 10:07 | |
*** daemontool has quit IRC | 10:07 | |
*** daemontool_ is now known as daemontool | 10:14 | |
*** vannif_ has quit IRC | 10:59 | |
*** emildi has joined #openstack-freezer | 11:09 | |
*** vannif has joined #openstack-freezer | 11:09 | |
daemontool | emildi, hi | 11:25 |
daemontool | :) | 11:25 |
emildi | hi daemontool :) | 11:43 |
daemontool | all we need to approve this https://review.openstack.org/#/c/270750/ and this https://review.openstack.org/#/c/270753/ as soon as they pass the checks | 12:01 |
*** daemontool has quit IRC | 13:19 | |
*** yangyapeng has quit IRC | 13:46 | |
*** EinstCrazy has quit IRC | 13:47 | |
*** daemontool has joined #openstack-freezer | 14:05 | |
*** ddieterly has joined #openstack-freezer | 15:26 | |
openstackgerrit | Fausto Marzi proposed openstack/freezer-web-ui: Align requirements to liberty global-requirements https://review.openstack.org/246981 | 15:31 |
*** dschroeder has joined #openstack-freezer | 15:35 | |
*** EinstCrazy has joined #openstack-freezer | 15:45 | |
*** pennerc has joined #openstack-freezer | 15:45 | |
ddieterly | does this meeting usually start late? | 16:05 |
ddieterly | is there a meeting today? | 16:08 |
ddieterly | daemontool: yo | 16:09 |
*** ddieterly has quit IRC | 16:11 | |
*** ddieterly has joined #openstack-freezer | 16:11 | |
ddieterly | hey, guys. is there a meeting today? | 16:12 |
ddieterly | dschroeder: hey, david. can you see my posts? | 16:13 |
dschroeder | Yes I can. | 16:14 |
ddieterly | ok, i thought maybe i had a connectivity problem | 16:14 |
ddieterly | isn't there supposed to be a meeting right now? | 16:15 |
ddieterly | dschroeder: isn't there supposed to be a meeting right now or did i get the times wrong? | 16:16 |
dschroeder | You got the times right, 4pm UTC on Thursdays. | 16:19 |
ddieterly | ok, thought i might have been mistaken | 16:19 |
ddieterly | anybody know where daemontool is? | 16:20 |
m3m0 | hey guys, sorry for the delay | 16:23 |
pennerc | Lets assume there is no meeting today. We do need to verify the meeting with Fausto. Currently Fausto is not on line, which would be the reason for no meeting. | 16:23 |
m3m0 | today I'm really really busy | 16:23 |
pennerc | or maybe he is? | 16:23 |
m3m0 | but the meeting is off for today, sorry for the short notice | 16:24 |
ddieterly | who usually runs the meeting? | 16:24 |
m3m0 | me | 16:24 |
ddieterly | why is it off? | 16:24 |
m3m0 | work issues | 16:24 |
ddieterly | m3m0: could we please get advanced warning if the meeting is going to be cancelled? | 16:27 |
m3m0 | for sure, my bad, I just had bad timing today | 16:27 |
daemontool | I'm here sorry I had a last minute important meeting | 16:34 |
daemontool | so we can still run 25 min | 16:35 |
daemontool | if folks are good | 16:35 |
*** frescof has joined #openstack-freezer | 16:39 | |
daemontool | I think we need to define e backup chariman for the meetings, in case the main cannot run the meeting | 16:42 |
daemontool | but I agree with ddieterly we need to let people know this before hand | 16:42 |
ddieterly | daemontool: any idea why my restore is failing on my dev box? | 16:49 |
ddieterly | looks like gzip is not happy | 16:50 |
ddieterly | do i need a certain version of gzip? | 16:50 |
*** ddieterly has quit IRC | 16:50 | |
*** ddieterly has joined #openstack-freezer | 16:51 | |
daemontool | ddieterly, gzip is backward compatible between versions | 16:52 |
daemontool | I don't think the issue is gzip | 16:52 |
daemontool | but I've been very busy with meetings and preparing the m2 release for mitaka | 16:52 |
daemontool | so I couldn't test it | 16:52 |
daemontool | vannif, ping | 16:52 |
ddieterly | 2016-01-20 16:17:12,370 root INFO Restore backup deklans-macbook-pro_backup-0_1453330051_0 | 16:55 |
ddieterly | 2016-01-20 16:17:12,386 root INFO [*] Pipe closed as EOF reached. Data transmitted successfully | 16:55 |
ddieterly | 2016-01-20 16:17:12,391 root ERROR [*] Restore error: gzip: (stdin): unexpected end of file | 16:55 |
ddieterly | /usr/local/bin/gtar: Child returned status 1 | 16:55 |
ddieterly | /usr/local/bin/gtar: Error is not recoverable: exiting now | 16:55 |
ddieterly | Traceback (most recent call last): | 16:55 |
ddieterly | File "/Users/dieterlyd/virtualenvs/freezer/lib/python2.7/site-packages/freezer/engine/tar/tar_engine.py", line 129, in restore_level | 16:55 |
ddieterly | tar_process.stdin.write(read_pipe.recv_bytes()) | 16:55 |
ddieterly | EOFError | 16:55 |
ddieterly | 2016-01-20 16:17:12,394 root CRITICAL [*] Critical Error: failed to restore file | 16:55 |
ddieterly | (i hope that is not 'flooding' the channel) | 16:56 |
daemontool | ddieterly, np | 16:56 |
daemontool | so after the parallal backup feature merge | 16:56 |
daemontool | I did tests | 16:57 |
daemontool | but using a config file | 16:57 |
daemontool | it shoult be the same tho | 16:57 |
daemontool | let me finish the branching for the freezer-web-ui | 16:57 |
daemontool | component | 16:57 |
daemontool | and be ready for m2 mitaka | 16:57 |
daemontool | and I'll check it | 16:57 |
daemontool | ddieterly, if you download that backup level 0 object form swift | 16:59 |
daemontool | can you tar gunzip it_ | 16:59 |
daemontool | ? | 16:59 |
ddieterly | i'll try it as soon as i get my devstack env back up | 17:00 |
daemontool | ok | 17:00 |
daemontool | so we need to define a second chair | 17:00 |
daemontool | for the meetings | 17:00 |
ddieterly | yea | 17:01 |
daemontool | or in general we can define that if the chair is not avail if one of the cores is online then he do it | 17:01 |
daemontool | the thing is that Eldar and Pierre are off | 17:01 |
daemontool | vannif < pinfg -s 1380 -f | 17:01 |
daemontool | s/pinfg/ping/ | 17:02 |
ddieterly | yea, i think that these meetings are critical from an openstack community perspective | 17:06 |
daemontool | ddieterly, yes did you wanted to discuss anything in particular? | 17:07 |
daemontool | anything I can help you with_ | 17:07 |
daemontool | ? | 17:07 |
ddieterly | just that problem i'm having ;-) | 17:07 |
daemontool | a part that error there that needs to be reproduced and debugged? | 17:07 |
daemontool | ok | 17:07 |
ddieterly | but i'm not the only person interested in freezer | 17:07 |
daemontool | ddieterly, I agree, are you ok to be the backup chairman? | 17:08 |
ddieterly | sure | 17:08 |
ddieterly | one question i do have is this | 17:08 |
ddieterly | has freezer been perf tested or load tested anywhere? | 17:08 |
ddieterly | is there an env where integration, perf, and load testing can be performed? | 17:09 |
ddieterly | i'm assuming this would be inside hpe | 17:09 |
m3m0 | hey guys vannif is around somewhere, we are having very busy schedules | 17:09 |
ddieterly | is anyone outside of hpe using freezer? | 17:09 |
daemontool | ddieterly, I did that in the hp public cloud | 17:09 |
ddieterly | is it a permanent env that is reusable? | 17:10 |
daemontool | like I think | 17:10 |
m3m0 | and I'll be out of the office next week, so ddieterly you will have to be the chairman next week | 17:10 |
daemontool | ddieterly, no, the hp public cloud does not exist anymore :) | 17:10 |
daemontool | but at least in the last 6 months | 17:10 |
daemontool | nothing about that was done | 17:10 |
daemontool | there was a performance test that was done few monts ago | 17:10 |
daemontool | months ago | 17:10 |
daemontool | it should be there in the hp Jira | 17:11 |
daemontool | it wasn't published as it was specific to Helion | 17:11 |
daemontool | thereis also a page called automation_backup in the hpcloud wiki with some other peformance report | 17:12 |
daemontool | but I think that tests should be done now with Rally | 17:12 |
daemontool | that's why I'm improving our dsvm gate job | 17:12 |
ddieterly | ok | 17:12 |
daemontool | but like high load tests are a different thing | 17:12 |
ddieterly | anyone outside of hpe using freezer? | 17:13 |
daemontool | I know someone from groupon was using it to backup mongodb | 17:13 |
daemontool | there's a roadmap to use it in Ericsson | 17:13 |
daemontool | and couple of other Telcos are using it | 17:13 |
ddieterly | how many devs do you plan to have at ericsson working on freezer? | 17:14 |
daemontool | I don't know | 17:14 |
daemontool | it depends on the business | 17:14 |
daemontool | as in any consultancy company | 17:14 |
ddieterly | are you wanting to remain ptl while your at ericsson? | 17:15 |
daemontool | until April yes | 17:15 |
daemontool | it's not about the company anyway | 17:15 |
daemontool | being ptl | 17:15 |
daemontool | is about your contribution to the project | 17:16 |
daemontool | in April there are the votation for ptl in openstack | 17:16 |
ddieterly | freezer is kind of your baby | 17:16 |
daemontool | on most of the services | 17:16 |
ddieterly | i thought that you might want to remain ptl | 17:16 |
daemontool | ddieterly, I was thinking about not propose myself for ptl once more | 17:17 |
daemontool | to give to others the opportunity | 17:17 |
daemontool | but the thing is, that the work needs to be done :) | 17:17 |
daemontool | it's about doing the work and make things better, not being ptl | 17:17 |
ddieterly | do any of the ps guys have an interest in being ptl? | 17:17 |
daemontool | being ptl is a consequence | 17:17 |
daemontool | ddieterly, I think is better if you ask to them directly as you work on the same environmebnt | 17:18 |
daemontool | but for sure | 17:18 |
daemontool | you need to be core contrib to apply as ptl | 17:18 |
daemontool | otherwise it doesn't make sense | 17:18 |
daemontool | that's the same for any os project | 17:19 |
daemontool | any core can apply for it | 17:19 |
daemontool | it's a self-application | 17:20 |
daemontool | ddieterly, I have some question too | 17:20 |
ddieterly | yea, just wondering what the interest level in contributing is from the current team members | 17:21 |
daemontool | if possible :) | 17:21 |
ddieterly | wondering if people want to be more or less involved | 17:21 |
ddieterly | wondering where the project is heading | 17:21 |
daemontool | I think it's a balance | 17:21 |
daemontool | of benefit for community | 17:21 |
ddieterly | sure, ask away | 17:21 |
daemontool | and for the business | 17:21 |
daemontool | so how many people from hp do you plan to allocate for freezer? | 17:22 |
ddieterly | well, just the team that is currently working on it so far | 17:22 |
ddieterly | and me ;-) | 17:22 |
daemontool | but that needs to be driven by the business | 17:22 |
daemontool | so if for hp | 17:22 |
daemontool | that make sense from the business perspective | 17:22 |
daemontool | people can be allocated | 17:22 |
daemontool | like for any other company | 17:23 |
daemontool | I think engineers wants to work 100% of their time on it | 17:23 |
ddieterly | there are no plans at this time to increase head count for freezer | 17:23 |
daemontool | ddieterly, it would be good to have at least the current one full time on it | 17:23 |
ddieterly | i'm not sure what the attitude of the ps guys is | 17:23 |
daemontool | s/one/ones/ | 17:23 |
daemontool | well that's an internal thing you need to solve internally | 17:24 |
ddieterly | do they want to work on it, or are they told to work on it | 17:24 |
ddieterly | yes, i am the current full-time guy | 17:24 |
daemontool | ddieterly, there's only one way to know it | 17:24 |
daemontool | :) | 17:24 |
ddieterly | i have just parachuted in | 17:24 |
ddieterly | lucky me ;-) | 17:24 |
daemontool | I think you are lucky | 17:24 |
daemontool | because you can work 100% of your time on upstream | 17:25 |
daemontool | people are not that lucky | 17:25 |
daemontool | and I think that's the issue the ps is having right now | 17:25 |
ddieterly | yes, but i have to leave my beloved monasca | 17:26 |
daemontool | well that is sad, I'm sorry | 17:26 |
daemontool | really | 17:26 |
daemontool | but that's an issue between you and your manager unfortunately | 17:26 |
ddieterly | :-( | 17:26 |
daemontool | sorry | 17:27 |
ddieterly | just kidding | 17:27 |
daemontool | so | 17:27 |
daemontool | what I think would be good | 17:27 |
daemontool | is that you organize some code walkthrough session | 17:27 |
daemontool | with reldan | 17:27 |
daemontool | and vannif | 17:27 |
daemontool | that helps a lot | 17:27 |
ddieterly | interesting idea | 17:27 |
daemontool | we did that in the past | 17:28 |
daemontool | and the outcome was good | 17:28 |
daemontool | for the web ui m3m0 can hep you | 17:28 |
daemontool | s/hep/help/ | 17:28 |
daemontool | what you like more? | 17:28 |
daemontool | writing code for the api? | 17:29 |
daemontool | web ui? | 17:29 |
daemontool | or the agent | 17:29 |
daemontool | freezer under some aspect is not that different than monasca | 17:29 |
ddieterly | i do not do ui work ;-) | 17:29 |
daemontool | ok | 17:30 |
ddieterly | only if i have to | 17:30 |
daemontool | web ui I think we are good | 17:30 |
daemontool | so | 17:30 |
daemontool | one thing I was working on | 17:30 |
daemontool | and I could finish | 17:30 |
daemontool | is the block based incremental | 17:30 |
daemontool | based on rsync | 17:30 |
daemontool | (the rsync algorithm not the tool) | 17:31 |
daemontool | it's a though bone | 17:31 |
daemontool | are you interested working on it_ | 17:31 |
daemontool | ? | 17:31 |
daemontool | that is needed | 17:31 |
ddieterly | sure, but anything i work on has to be in alignment with hpe plans | 17:31 |
ddieterly | i'm not free to decide | 17:32 |
daemontool | I agree | 17:32 |
m3m0 | sudo ethtool -i eth0 | 17:32 |
daemontool | I think that make sense for hp | 17:32 |
m3m0 | sorry wrong windows | 17:32 |
daemontool | m3m0, sure | 17:32 |
daemontool | haha | 17:32 |
daemontool | so | 17:32 |
daemontool | block based backup allows openstack | 17:32 |
daemontool | to retain data for longer time | 17:32 |
ddieterly | so, am i officially the back up chair now? | 17:32 |
daemontool | use less band | 17:32 |
daemontool | ddieterly, I think so yes | 17:33 |
ddieterly | do we need to vote or something like that? | 17:33 |
ddieterly | oh, question | 17:33 |
daemontool | nope I'm going to add it to the wiki | 17:33 |
ddieterly | was the logging level left out of the documentation on purpose or was that just an oversight? | 17:33 |
ddieterly | see https://review.openstack.org/#/c/266552/ | 17:34 |
ddieterly | other question, why was elasticsearch chosen as db? | 17:35 |
ddieterly | what was the compelling reason(s) | 17:35 |
daemontool | I think is wan't added to the README when the feature was implemented | 17:35 |
daemontool | oversight | 17:35 |
daemontool | there were 2 reason | 17:36 |
ddieterly | k | 17:36 |
daemontool | 1) the data generated was always json | 17:36 |
daemontool | so there's no additional computation required, just the API recieve the data from the agent/scheduler | 17:36 |
daemontool | and store it to the es | 17:36 |
daemontool | cause the data we manage is json mainly | 17:37 |
ddieterly | i see | 17:37 |
daemontool | 2) with es all the replica, ha, sharding mechanism | 17:37 |
daemontool | is done automatically | 17:37 |
ddieterly | nice | 17:37 |
daemontool | you do not need to set it up like in mysql | 17:37 |
daemontool | so lower the operational costs | 17:37 |
daemontool | but | 17:37 |
ddieterly | is anyone else using pycharm for development? | 17:37 |
daemontool | yes all of us | 17:37 |
ddieterly | with virtual envs? | 17:38 |
daemontool | nope | 17:38 |
*** samuelBartel has quit IRC | 17:38 | |
daemontool | I use a new vm with sshfs | 17:38 |
daemontool | for most of the things | 17:38 |
daemontool | but yes | 17:38 |
daemontool | also many other times I use | 17:38 |
daemontool | pycharm with virtualenvs | 17:38 |
ddieterly | ok | 17:38 |
daemontool | ddieterly, re db | 17:39 |
daemontool | but after a bit | 17:39 |
daemontool | we thought it was good to have also another backend as db | 17:39 |
daemontool | like mysql | 17:39 |
daemontool | but no one implemented it yes | 17:39 |
daemontool | if you want to do it | 17:39 |
daemontool | or you are interested | 17:39 |
daemontool | also that's another thing to do | 17:40 |
ddieterly | no shortage of things to do | 17:40 |
daemontool | not at all | 17:40 |
daemontool | ;) | 17:40 |
ddieterly | right now, hpe is eager to get elasticsearch backed up | 17:40 |
daemontool | ok | 17:41 |
daemontool | you have lvm in helion | 17:41 |
daemontool | so you can do the backup | 17:42 |
daemontool | the best person to ask for that is Slashme but currently he's off too | 17:42 |
ddieterly | there is a swift plugin for es | 17:42 |
daemontool | ok | 17:42 |
daemontool | you don't need to backup an individual index right? | 17:42 |
ddieterly | right | 17:43 |
ddieterly | https://github.com/wikimedia/search-repository-swift | 17:43 |
daemontool | just the whole thing? | 17:43 |
ddieterly | yes | 17:43 |
daemontool | then use the freezer-agent with lvm snapshots | 17:43 |
daemontool | and that should work | 17:43 |
ddieterly | i am trying to get this plugin to work | 17:43 |
daemontool | ok | 17:43 |
ddieterly | does that snapshot and the backup | 17:43 |
daemontool | yes | 17:44 |
daemontool | lvm snapshot | 17:44 |
daemontool | backup of let's say /var/lib/elasticsearch | 17:44 |
daemontool | remove the snapshot | 17:44 |
daemontool | done | 17:44 |
daemontool | the problem is the space | 17:44 |
ddieterly | the plugin does the snapshot and the backup | 17:44 |
daemontool | ah ok | 17:44 |
daemontool | then use that if you feel more comfortable | 17:45 |
ddieterly | if we can get it to work, then it will be best choice for now | 17:45 |
daemontool | ok | 17:45 |
daemontool | why do you think is the best choice? | 17:45 |
daemontool | it can do incrementals? | 17:45 |
daemontool | if it can do incrementals then yes | 17:48 |
daemontool | probably is | 17:48 |
daemontool | but I think it depends also on the dataset size | 17:49 |
*** arunb has joined #openstack-freezer | 17:49 | |
ddieterly | yes, it does incrementals | 17:52 |
daemontool | I'd check how it perform on the dataset you need to backup | 17:53 |
ddieterly | yea, that's why i'm interested in scale and perf testing | 17:55 |
daemontool | the bottleneck there would be es itself | 17:56 |
daemontool | as you are extracting the data using es | 17:56 |
daemontool | while freezer backup the files on the fs | 17:56 |
ddieterly | the plugin puts the backup directly into swift | 17:57 |
daemontool | so you need to make an http request to an endpoint and it will trigger the backup? | 17:58 |
daemontool | I mean to an es endpoint | 17:59 |
daemontool | or does it follow its own schedule? | 18:00 |
daemontool | one thing to keep in mind is that putting that load on es for the backup, will affect the logging service itself | 18:04 |
openstackgerrit | Fausto Marzi proposed openstack/freezer: Add check for name=='__main__' https://review.openstack.org/266552 | 18:09 |
daemontool | ddieterly, sorry | 18:09 |
daemontool | my bad there | 18:09 |
openstackgerrit | Fausto Marzi proposed openstack/freezer: Add check for name=='__main__' https://review.openstack.org/266552 | 18:11 |
daemontool | please disregard patchset 8 there | 18:11 |
daemontool | did a git fetch and check out from the wrong commit sorry | 18:12 |
ddieterly | np | 18:12 |
ddieterly | does the horizion ui have a way to download the back up files? | 18:13 |
ddieterly | i can download the metadata file, but the actual backup does not seem to have a way to download it thru the ui | 18:13 |
ddieterly | ah, i see, you have go deep in the tree to get to the actual file | 18:15 |
ddieterly | daemontool: i was able to backup a file to swift using freezerc, then download it with the horizon ui, then untar it and look at it | 18:17 |
daemontool | ok | 18:17 |
ddieterly | so, the file seems good in swift | 18:17 |
daemontool | well you can download it with swift cli | 18:18 |
daemontool | yes | 18:18 |
daemontool | so the problem is definetly not tar or gzip | 18:18 |
daemontool | but I need to reproduce | 18:18 |
daemontool | the issue | 18:18 |
ddieterly | k | 18:19 |
*** arunb has quit IRC | 18:20 | |
openstackgerrit | Fausto Marzi proposed openstack/freezer-web-ui: Align requirements to liberty global-requirements https://review.openstack.org/246981 | 18:25 |
*** daemontool has quit IRC | 18:35 | |
ddieterly | does freezer api require a specific version of elasticsearch | 18:52 |
ddieterly | freezer api devstack uses 1.4.2 | 18:53 |
ddieterly | current version of elastic search is 2.1.1 | 18:53 |
*** daemontool has joined #openstack-freezer | 19:42 | |
daemontool | ddieterly, I'm here available for ~1 hr if you need anything | 19:50 |
openstackgerrit | Fausto Marzi proposed openstack/freezer-web-ui: Align requirements to liberty global-requirements https://review.openstack.org/246981 | 20:00 |
*** daemontool has quit IRC | 20:01 | |
*** daemontool has joined #openstack-freezer | 20:02 | |
*** daemontool_ has joined #openstack-freezer | 20:15 | |
*** daemontool has quit IRC | 20:17 | |
ddieterly | daemontool_: just show your message | 20:17 |
ddieterly | you still there and for how long? | 20:17 |
daemontool_ | 20 min | 20:27 |
*** daemontool_ has quit IRC | 20:52 | |
*** daemontool_ has joined #openstack-freezer | 20:53 | |
*** arunb has joined #openstack-freezer | 20:53 | |
daemontool_ | Hi arunb | 20:57 |
arunb | Hi | 20:57 |
daemontool_ | how are you doing :) | 20:57 |
arunb | I am good. Who is daemontool? I assume its Fausto :) | 20:57 |
daemontool_ | this is Fausto | 20:57 |
arunb | how are you | 20:57 |
daemontool_ | yes | 20:57 |
daemontool_ | not too bad | 20:57 |
arunb | Its late in Italy. Still working? | 20:58 |
daemontool_ | always working | 20:58 |
openstackgerrit | Merged openstack/freezer: Align requirements and version to Mitaka https://review.openstack.org/269235 | 21:12 |
openstackgerrit | Merged openstack/freezer-api: Align requirements and version to Mitaka https://review.openstack.org/269237 | 21:12 |
*** ddieterly has quit IRC | 21:36 | |
openstackgerrit | Merged openstack/freezer-web-ui: Align requirements to liberty global-requirements https://review.openstack.org/246981 | 21:56 |
*** pennerc has quit IRC | 22:51 | |
daemontool_ | https://review.openstack.org/#/c/271070/ | 22:55 |
daemontool_ | hope we'll make it for mitaka | 22:55 |
daemontool_ | :) | 22:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!