*** fungi has quit IRC | 00:08 | |
*** DennyZha` has joined #openstack-infra | 00:10 | |
*** DennyZhang has quit IRC | 00:12 | |
*** DennyZha` has quit IRC | 00:15 | |
*** michchap has joined #openstack-infra | 00:21 | |
*** sarob has joined #openstack-infra | 00:25 | |
*** sarob_ has joined #openstack-infra | 00:28 | |
*** sarob has quit IRC | 00:29 | |
*** sarob_ has quit IRC | 00:33 | |
*** dims has quit IRC | 00:45 | |
*** sarob has joined #openstack-infra | 00:47 | |
*** dims has joined #openstack-infra | 00:49 | |
*** sarob has quit IRC | 00:51 | |
*** dims has quit IRC | 00:54 | |
*** yongli has quit IRC | 01:01 | |
*** kpepple_ has joined #openstack-infra | 01:01 | |
*** dripton_ has joined #openstack-infra | 01:02 | |
*** fungi has joined #openstack-infra | 01:02 | |
*** Steap_ has joined #openstack-infra | 01:03 | |
*** ekarlso- has joined #openstack-infra | 01:06 | |
*** fungi has quit IRC | 01:08 | |
*** ekarlso has quit IRC | 01:08 | |
*** Steap has quit IRC | 01:08 | |
*** terriyu has quit IRC | 01:08 | |
*** dripton has quit IRC | 01:08 | |
*** jpeeler has quit IRC | 01:08 | |
*** rwsu has quit IRC | 01:08 | |
*** kpepple has quit IRC | 01:08 | |
*** rwsu has joined #openstack-infra | 01:14 | |
*** jpeeler has joined #openstack-infra | 01:15 | |
*** terriyu has joined #openstack-infra | 01:15 | |
*** fungi has joined #openstack-infra | 01:16 | |
*** michchap has quit IRC | 01:25 | |
*** michchap has joined #openstack-infra | 01:28 | |
*** sarob has joined #openstack-infra | 01:47 | |
*** sarob has quit IRC | 01:52 | |
*** wenlock has joined #openstack-infra | 02:01 | |
pleia2 | terriyu: we have some ruby stuff related to puppet on the back burner, but not enough for a full internship and it's sysadmin focused (so ruby is helpful but not a focus) | 02:37 |
---|---|---|
terriyu | thanks pleia2 | 02:38 |
terriyu | we got ahold of anteaya and she said something similar | 02:38 |
* pleia2 nods | 02:38 | |
*** changbl has quit IRC | 02:47 | |
*** sarob has joined #openstack-infra | 02:48 | |
*** sarob has quit IRC | 02:53 | |
*** nati_ueno has joined #openstack-infra | 02:59 | |
*** changbl has joined #openstack-infra | 03:02 | |
*** esheffield has quit IRC | 03:07 | |
*** lascii has quit IRC | 03:07 | |
*** esheffield has joined #openstack-infra | 03:11 | |
*** dcramer_ has joined #openstack-infra | 03:18 | |
*** esheffield has quit IRC | 03:20 | |
*** sarob has joined #openstack-infra | 03:48 | |
*** sarob has quit IRC | 03:53 | |
*** terriyu has quit IRC | 04:09 | |
*** wenlock has quit IRC | 04:20 | |
*** lnxnut has quit IRC | 04:26 | |
*** lnxnut has joined #openstack-infra | 04:31 | |
*** reed has quit IRC | 04:41 | |
*** sarob has joined #openstack-infra | 04:49 | |
*** sarob has quit IRC | 04:53 | |
*** dcramer_ has quit IRC | 04:55 | |
*** esheffield has joined #openstack-infra | 04:57 | |
*** ArxCruz has quit IRC | 04:58 | |
*** ArxCruz has joined #openstack-infra | 05:02 | |
*** esheffield has quit IRC | 05:06 | |
*** zojoncj has joined #openstack-infra | 05:24 | |
*** sarob has joined #openstack-infra | 05:49 | |
*** sarob has quit IRC | 05:54 | |
*** moted has quit IRC | 06:19 | |
*** SergeyLukjanov has joined #openstack-infra | 06:24 | |
*** sarob has joined #openstack-infra | 06:50 | |
*** tkammer has joined #openstack-infra | 06:53 | |
*** sarob has quit IRC | 06:55 | |
*** tkammer has quit IRC | 07:18 | |
*** ericw has joined #openstack-infra | 07:18 | |
*** ericw has quit IRC | 07:23 | |
*** nati_ueno has quit IRC | 07:24 | |
*** nati_ueno has joined #openstack-infra | 07:24 | |
*** nati_ueno has quit IRC | 07:29 | |
*** SergeyLukjanov has quit IRC | 07:39 | |
*** SergeyLukjanov has joined #openstack-infra | 07:46 | |
*** pvo has quit IRC | 07:48 | |
*** SergeyLukjanov_ has joined #openstack-infra | 07:48 | |
*** boris-42 has quit IRC | 07:48 | |
*** fifieldt has joined #openstack-infra | 07:48 | |
*** pvo has joined #openstack-infra | 07:50 | |
*** sarob has joined #openstack-infra | 07:50 | |
*** SergeyLukjanov has quit IRC | 07:51 | |
*** ArxCruz has quit IRC | 07:51 | |
*** sarob has quit IRC | 07:55 | |
*** SergeyLukjanov_ has quit IRC | 08:00 | |
*** tkammer has joined #openstack-infra | 08:19 | |
*** tkammer_ has joined #openstack-infra | 08:27 | |
*** tkammer has quit IRC | 08:31 | |
lifeless | I want to stab python 2's confused unicode handling so badly | 08:37 |
lifeless | stabbity stabbity stab stab stab | 08:37 |
fifieldt | +1 | 08:44 |
*** sarob has joined #openstack-infra | 08:51 | |
*** rakhmerov has joined #openstack-infra | 08:55 | |
*** sarob has quit IRC | 08:55 | |
*** rakhmerov has quit IRC | 08:56 | |
*** dkliban has joined #openstack-infra | 09:09 | |
*** rakhmerov has joined #openstack-infra | 09:40 | |
*** rakhmerov has quit IRC | 09:45 | |
lifeless | and yay, Python3 compat fixed for crcache. | 09:48 |
*** marun has joined #openstack-infra | 09:51 | |
*** sarob has joined #openstack-infra | 09:52 | |
*** sarob has quit IRC | 10:00 | |
*** rakhmerov has joined #openstack-infra | 10:10 | |
*** ericw has joined #openstack-infra | 10:18 | |
*** Bada has joined #openstack-infra | 10:22 | |
*** ericw has quit IRC | 10:23 | |
*** Bada has quit IRC | 10:38 | |
*** rakhmerov has quit IRC | 10:44 | |
*** marun_ has joined #openstack-infra | 11:08 | |
*** alexpilotti has joined #openstack-infra | 11:09 | |
*** marun has quit IRC | 11:10 | |
*** marun_ is now known as marun | 11:10 | |
*** fbo_away is now known as fbo | 11:15 | |
*** boris-42 has joined #openstack-infra | 11:15 | |
*** rakhmerov has joined #openstack-infra | 11:40 | |
*** rakhmerov has quit IRC | 11:44 | |
marun | lifeless: ping | 11:49 |
*** rakhmerov has joined #openstack-infra | 11:56 | |
*** sarob has joined #openstack-infra | 11:57 | |
*** rakhmerov has quit IRC | 12:00 | |
*** sarob has quit IRC | 12:01 | |
*** michchap_ has joined #openstack-infra | 12:06 | |
*** michchap has quit IRC | 12:06 | |
*** fifieldt has quit IRC | 12:09 | |
*** annegentle_ has quit IRC | 12:23 | |
openstackgerrit | A change was merged to openstack-infra/os-loganalyze: add tempest log to the list of level supporting logs https://review.openstack.org/53439 | 12:53 |
*** rakhmerov has joined #openstack-infra | 12:57 | |
*** davidhadas_ has joined #openstack-infra | 13:16 | |
*** davidhadas has quit IRC | 13:17 | |
*** sarob has joined #openstack-infra | 13:28 | |
*** rakhmerov has quit IRC | 13:31 | |
*** sarob has quit IRC | 13:32 | |
*** ericw has joined #openstack-infra | 14:13 | |
*** dims has joined #openstack-infra | 14:16 | |
*** rakhmerov has joined #openstack-infra | 14:25 | |
*** sarob has joined #openstack-infra | 14:28 | |
*** sarob has quit IRC | 14:32 | |
*** sarob has joined #openstack-infra | 14:39 | |
*** anteaya has joined #openstack-infra | 15:00 | |
*** tkammer_ has quit IRC | 15:28 | |
*** tkammer has joined #openstack-infra | 15:28 | |
*** tkammer has quit IRC | 15:31 | |
*** Bada has joined #openstack-infra | 15:33 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 15:41 |
anteaya | don't bother looking yet, still much to do | 15:42 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 15:54 |
*** changbl has quit IRC | 15:55 | |
*** fbo is now known as fbo_away | 15:56 | |
*** marun has quit IRC | 15:56 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 15:58 |
*** Steap_ is now known as Steap | 16:00 | |
*** Bada has quit IRC | 16:01 | |
*** ericw has quit IRC | 16:24 | |
*** oubiwann has joined #openstack-infra | 16:27 | |
*** tkammer has joined #openstack-infra | 16:39 | |
*** tkammer_ has joined #openstack-infra | 16:41 | |
*** dims has quit IRC | 16:42 | |
*** tkammer has quit IRC | 16:44 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 16:49 |
*** fbo_away is now known as fbo | 16:51 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 16:53 |
*** dkliban has quit IRC | 16:54 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 17:03 |
*** SergeyLukjanov has joined #openstack-infra | 17:08 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 17:08 |
anteaya | if any bored, kind soul feels like giving 54030 a pass for syntax and suggestions on what the root directory of this app should be, I'm grateful | 17:10 |
* anteaya goes for a walk in the sunshine | 17:10 | |
*** SergeyLukjanov has quit IRC | 17:13 | |
*** SergeyLukjanov_ has joined #openstack-infra | 17:13 | |
*** SergeyLukjanov_ has quit IRC | 17:24 | |
*** Bada has joined #openstack-infra | 17:26 | |
*** SergeyLukjanov has joined #openstack-infra | 17:31 | |
*** SergeyLukjanov is now known as _SergeyLukjanov | 17:35 | |
*** _SergeyLukjanov has quit IRC | 17:36 | |
*** tkammer_ has quit IRC | 17:36 | |
*** SergeyLukjanov has joined #openstack-infra | 17:49 | |
*** dcramer_ has joined #openstack-infra | 17:50 | |
*** pabelanger has quit IRC | 17:50 | |
*** reed has joined #openstack-infra | 17:58 | |
*** rakhmerov has quit IRC | 17:59 | |
*** pabelanger has joined #openstack-infra | 18:03 | |
*** che-arne has quit IRC | 18:06 | |
pleia2 | anteaya: my review isn't exceptionally thorough, but I'll be able to see more clearly once the things I noted are looked at :) | 18:19 |
mordred | anteaya: I also recommend potentially getting youreslf set up to be able to run the puppet code on a cloud server | 18:21 |
mordred | anteaya: that way you can test locally and ensure it's doing what you think it should do | 18:21 |
mordred | the time spent to grok how that works is probably well worth it in the long run | 18:21 |
pleia2 | ++ | 18:21 |
pleia2 | during the sprint in NYC we fixed up http://ci.openstack.org/sysadmin.html#making-a-change-in-puppet so it should be accurate for testing changes on a cloud server | 18:22 |
clarkb | I really like that the summit sessions don't seem to be nova day one, neutron day two, all other projects day three then QA/process/infra day four | 18:22 |
*** rakhmerov has joined #openstack-infra | 18:23 | |
pleia2 | clarkb: yeah | 18:23 |
sdague | clarkb: yeh, though still with so many tracks, it's going to be tough. | 18:23 |
sdague | at least ttx didn't overlap qa and infra this time :) that was kind of rough | 18:24 |
lifeless | o/ | 18:24 |
*** SergeyLukjanov has quit IRC | 18:25 | |
*** dkliban has joined #openstack-infra | 18:27 | |
mordred | clarkb: ++ | 18:28 |
sdague | I'm assuming we'll talk more more generic logstash/elasticsearch things in the elastic-recheck session. Been doing experiments to figure out the limits of what we can get out of it for those discussions. | 18:29 |
*** che-arne has joined #openstack-infra | 18:30 | |
clarkb | sdague: ++ I am really excited to hear feedback from the query side. I feel like I spend so much time making it go that I haven't had a chance to play with the query API | 18:30 |
clarkb | I think nodepool is having a hard time spinning up rackspace nodes for that d-g change. We should allow the d-g jobs to run on hpcloud too and look at nodepool logs | 18:31 |
clarkb | fungi: ^ | 18:31 |
sdague | yeh, there are some really neat things you can get out of it per - http://dague.net/2013/10/27/openstack-ci-by-the-numbers/ - at the same time I'm completely unable to get other answers out of it that I'd like (I think basically because there is no server side reduce opperation) | 18:32 |
* mordred is very happy that there is a session on CI/CD Automation for TripleO the same slot as the Updates to hacking oslo session he's running | 18:32 | |
sdague | heh | 18:32 |
sdague | yeh, I noticed that this morning | 18:32 |
sdague | the summit schedule looked so nice on thursday | 18:33 |
sdague | when only 1/4 of talks were out there | 18:33 |
dhellmann | mordred: I might be able to juggle that hacking session. I was trying to schedule you around the infra slots that conflict with oslo | 18:33 |
clarkb | sdague: woah! that set of numbers is neat | 18:33 |
mordred | dhellmann: it's ok - clark and jim can probably handle the TripleO CI/CD session | 18:34 |
clarkb | half a million qemu guests (I think they aren't technically kvm) | 18:34 |
mordred | I know which bits of hacking jeblair hates already | 18:34 |
dhellmann | mordred: ok | 18:34 |
sdague | clarkb: bah, sure :) | 18:34 |
clarkb | sdague: I prefer graphite for collecting metrics like that, but elasticsearch is handy because you don't ahve to preconfigure the thing you need numbers on | 18:35 |
clarkb | much more flexibility | 18:35 |
sdague | honestly, graphite still makes no sense to me. Maybe I need to sit down with you at some point to figure it out. | 18:36 |
mordred | sdague: ++ | 18:37 |
mordred | sdague: when I want a graphite graph, I just mention it offhand and hope that clarkb or jeblair will make it for me | 18:37 |
lifeless | mordred: btw today is a public holiday here | 18:37 |
mordred | lifeless: ossum | 18:37 |
lifeless | mordred: if I had not mentioned it | 18:37 |
clarkb | sdague: mordred: I find myself going to the graphs in zuul, copying their urls and working from there. If you open in firefox you get all of the special characters in ascii rather than escape codes which makes it much easier | 18:38 |
sdague | clarkb: yeh, what I find, I think, is I've got pretty strong data model / query notions in my head from years of sql. And spent a lot of time with matplotlib in the last year. And graphite's ideas of both series and data query are different enough I'm just always with a square peg and round hole. | 18:42 |
*** rakhmerov has quit IRC | 18:53 | |
*** rakhmerov has joined #openstack-infra | 18:54 | |
*** dkliban has quit IRC | 18:55 | |
mordred | dhellmann: actually, could you swap the tuesday python-openstackclient session with the hacking session? | 18:57 |
dhellmann | mordred: let me look | 18:57 |
mordred | I have no sessions I need to go to that chunk | 18:57 |
dhellmann | yeah, I can do that | 18:58 |
*** rakhmerov has quit IRC | 18:58 | |
dhellmann | mordred: done, but it may take a little bit before sched reflects the changes | 19:00 |
*** Bada has quit IRC | 19:00 | |
mordred | dhellmann: thanks! | 19:02 |
mordred | sdague, jgriffith: there is a cinder talk on rolling upgrades and a QA talk on rolling upgrades at the exact same time | 19:02 |
mordred | sdague, jgriffith: they may be tlaking about different things, but it's possible that they might be related :) | 19:03 |
sdague | heh, nice | 19:03 |
* dhellmann pictures the next summit being a month-long retreat to allow for all sessions to be in a single track | 19:03 | |
*** fbo is now known as fbo_away | 19:03 | |
sdague | so they are, the QA session is about nova in gate testing for this | 19:03 |
sdague | at least upgrade is getting plenty of attention at this summit | 19:06 |
*** fbo_away is now known as fbo | 19:07 | |
clarkb | if the d-g change is still stuck tomorrow, I think a ninja fix for it may be to add the label it wants to some unused hpcloud instances | 19:07 |
clarkb | just trick into running the tests there | 19:08 |
mordred | sdague:, dhellmann: yeah - I still think we're missing the big-picture track | 19:09 |
mordred | for things like "hey, all the cores and all the ptls, let's talk for a few minutes all in the same place about rolling upgrades/CD" | 19:09 |
sdague | mordred: how can you have a big picture track, competing against *6* project tracks | 19:09 |
mordred | sdague: that's just the thing - I think we can't | 19:09 |
mordred | but - I think we're getting bit enough that we really need one | 19:10 |
clarkb | I think you need a big picture track | 19:10 |
mordred | you need one that's exclusive | 19:10 |
clarkb | that doesn't conflcit with anything else | 19:10 |
clarkb | right | 19:10 |
mordred | ++ | 19:10 |
mordred | not many sessions, because the cost of each big-picture session is pretty high | 19:10 |
mordred | or - maybe we just need to get better about pre-coordinating some of those topics at the TC level | 19:10 |
sdague | yeh, that might be another option | 19:11 |
mordred | and then getting the individual projects to work on how to accomplish the stuff the tc/ptls work out should be overall project direction | 19:11 |
mordred | so we can say things lke "yup. rolling upgrades/continuous deploying is important" - so that each project doesn't have to decide the if | 19:11 |
mordred | I dunno | 19:12 |
mordred | it's hard | 19:12 |
mordred | I don't want to centralize things or anything | 19:12 |
clarkb | I think most people seem to agree on the if. Sorting out how is hard | 19:12 |
sdague | honestly, a lot of these things also can just go to the list. That's why I didn't bother trying to do the log harmonization thing as a summit thing, because you just needed more people in the thread | 19:12 |
mordred | my friday schedule isn't silly! | 19:12 |
mordred | sdague: ++ | 19:12 |
mordred | clarkb: yah | 19:12 |
mordred | there's actually two contiguous sessions on friday that do not require my attention! | 19:13 |
sdague | mordred: only 1/2 the schedule is up | 19:13 |
*** reed has quit IRC | 19:13 | |
mordred | sdague: oh dear god. you're kidding | 19:13 |
sdague | we have 6 tracks | 19:13 |
mordred | and here I was getting excited | 19:13 |
clarkb | speaking of upgrade testing. what if grenade started on a VM that already had a cloud running at the base version? | 19:13 |
sdague | I see only 4 blocks max in each session | 19:13 |
clarkb | sdague: in theory we could have nodepool build those VMs for us then we only need to worry about upgrade and test in grenade and not setup tests upgrade test | 19:14 |
sdague | clarkb: it would have running services by the time we start? | 19:14 |
sdague | is there an advantage on nodepool doing that vs. grenade? | 19:15 |
clarkb | sdague: yes have everything going on the node by the time zuul runs a test on it. Advantage would be simplifying logic in grenade possibly. Grenade doesn't need to know how to set up the various base clouds instead it would just know how to upgrade from them | 19:16 |
sdague | clarkb: remember, and important piece of grenade is the javelin piece, which creates a sample project and resources, so it's not an empty environment that we upgrade | 19:16 |
*** rcleere has joined #openstack-infra | 19:16 | |
sdague | I guess, it would completely tie grenade to the zuul env though. it is kind of nice to be able to just run it | 19:16 |
clarkb | sdague: right that could happen at the beginning (it is just an idea that occurred to me as it seems grenade spends a lot of time dealing with "known" good) | 19:16 |
clarkb | sdague: good point | 19:17 |
*** ericw has joined #openstack-infra | 19:17 | |
mordred | sdague: well, if grenade had phases | 19:18 |
mordred | then we could just have the nodepool piece run phase 1, the gate run phase 2 | 19:18 |
mordred | and a local run could run both | 19:18 |
sdague | I'm not sure we'd save a lot of grenade code doing that, it's all devstack lib calls. But it would be worth chatting about maybe. dansmith had some other thoughts about grenade upgrade scripts being extracted. | 19:18 |
* mordred waves hands | 19:18 | |
clarkb | sdague: grenade is very induction like, and I was thinking that we should treat the base version as "true" | 19:18 |
clarkb | rather than going through the steps to build and test it | 19:18 |
mordred | yeah. since we tested the base version elsewhere already | 19:18 |
sdague | clarkb: sure, I just wonder if we'd find how sensitive things are to initial conditions | 19:18 |
sdague | mordred: with what config... :) | 19:19 |
lifeless | actually, I should ask here | 19:19 |
lifeless | whats grokmirror ? | 19:19 |
lifeless | how is it different to mr? or config-manager, if that supported git? | 19:19 |
sdague | anyway, I'm being called off to yard tasks, back in a bit | 19:19 |
*** Bada has joined #openstack-infra | 19:19 | |
mordred | sdague: nod | 19:19 |
mordred | lifeless: grokmirror is a thign that linux kernel folks wrote | 19:20 |
mordred | you run it on your git servers - so git.openstack.org - | 19:20 |
mordred | and then you can use the grokmirror client to mirror in the set of repos | 19:20 |
mordred | https://www.linux.com/news/featured-blogs/203-konstantin-ryabitsev/720132-set-up-your-own-git-mirror-with-new-grokmirror-tool | 19:20 |
mordred | https://github.com/mricon/grokmirror | 19:20 |
mordred | lifeless: what are mr and config-manager ? | 19:21 |
lifeless | mordred: apt-cache show $1 | 19:22 |
lifeless | thanks, I see the difference w.r.t. grokmirror | 19:22 |
mordred | woot! | 19:23 |
mordred | ah. mr looks pretty cool | 19:23 |
fungi | sdague: mordred: i think pre-setup for the starting point of grenade may not be viable 100% of the time... since we should also use grenade to test that proposed stable branch patches are still creating an environment which can be upgraded to the next stable branch or (if most recent stable branch) tip of master | 19:27 |
mordred | fungi: that's an excellent point | 19:28 |
lifeless | mordred: config-manager predates mr by oh 6 or more years | 19:29 |
clarkb | so in theory induction still works in that case | 19:29 |
clarkb | mordred: fungi: you still assume the truth of the base | 19:29 |
lifeless | mordred: but I haven't maintained it in nearly that long :- I haven't added git support | 19:29 |
mordred | clarkb: not if your change is a change to base | 19:29 |
clarkb | mordred: in maths you would, but this isn't maths | 19:29 |
fungi | clarkb: well, agreed. induction of the base is fine since we have separate tests confirming that anyway | 19:29 |
mordred | clarkb: forward grenade tests test new change to base against assumed already good next version | 19:30 |
mordred | clarkb: so, a change to stable/havana gets tested for its upgradabiliy to trunk | 19:30 |
mordred | in that case, you cannot assume base is good at the time you're running the test | 19:30 |
clarkb | mordred: right, and if grenade runs and says trunk didn't break you have still done induction | 19:30 |
clarkb | mordred: you can, as you use N+1 to see if it falls apart | 19:30 |
clarkb | the problem here is that might not be reliable enough for our case | 19:31 |
mordred | yah. ok. off to the airport | 19:31 |
mordred | because, you know, that's what I do | 19:31 |
clarkb | mordred: you should come to seattle some time and see the new office | 19:31 |
mordred | clarkb: yah | 19:32 |
*** Bada has quit IRC | 19:33 | |
anteaya | pleia2: thanks | 19:34 |
anteaya | mordred: I can work on that, I don't think I can do that before summit though | 19:35 |
anteaya | and my belief I am holding is that you want this available for the board by summit | 19:35 |
anteaya | correct me if I am in error | 19:35 |
* anteaya makes some food and continues to read the backscroll | 19:35 | |
mordred | I do - BUT - I think being able to run a puppet command locally will let you iterate and grok more quickly | 19:36 |
mordred | anteaya: I could be wrong - but I have a hunch that if you can cycle on "oh, this change causes that result" and see it happening, the puppet crazy will click sooner | 19:36 |
anteaya | okay, i can head in that direction | 19:37 |
openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Run devstack-gate stable jobs on all providers https://review.openstack.org/54033 | 19:37 |
fungi | clarkb: ^ | 19:37 |
anteaya | I have no estimate if the puppet crazy will click before summit though | 19:37 |
anteaya | but here is hoping | 19:37 |
mordred | anteaya: well, it'll be easier now that you have a cloud account | 19:40 |
mordred | whereas before, you know, you did not so much have one of those | 19:40 |
anteaya | that is true | 19:40 |
anteaya | all hail the cloud account | 19:40 |
*** rcleere has quit IRC | 19:40 | |
clarkb | fungi: +2 from me. probably safe to approve that when you have time | 19:41 |
lifeless | anteaya: as a recent learner of puppet crazy | 19:41 |
fungi | clarkb: doing and monitoring | 19:41 |
* anteaya goes to watch toaster so the second piece doesn't set off the fire alarm | 19:41 | |
lifeless | anteaya: bringing up a full clone of openstack-infra and learning by doing was fairly essential | 19:41 |
lifeless | anteaya: there's lots of deeply coupled stuff in puppet | 19:41 |
lifeless | anteaya: like where things are found, that it takes a while to internalise | 19:42 |
anteaya | yes | 19:42 |
anteaya | so given this is Sunday | 19:42 |
lifeless | meh, you ancients | 19:42 |
lifeless | Monday, it's Monday | 19:42 |
anteaya | and I need Thursday for errands like laundry and banking | 19:42 |
anteaya | and Friday for travel | 19:42 |
anteaya | I have 3.5 days to be effective | 19:43 |
anteaya | the goal I had been shooting for was make mordred happy | 19:43 |
openstackgerrit | A change was merged to openstack-infra/config: Run devstack-gate stable jobs on all providers https://review.openstack.org/54033 | 19:43 |
lifeless | oh well that goal is easy | 19:43 |
lifeless | mordred: your next drink is on anteaya :) | 19:43 |
lifeless | tada. | 19:43 |
anteaya | but I can realign and do the clone of -infra | 19:43 |
anteaya | ha ha ha | 19:43 |
anteaya | okay whew | 19:43 |
lifeless | anteaya: oh! seriously, I wasn't saying do the clone of -infra thing, just do the local puppet recipe in a cloud server. | 19:44 |
lifeless | anteaya: thats what pleia2 pointed you at | 19:44 |
anteaya | oh okay | 19:44 |
anteaya | let me eat and then try again | 19:44 |
fungi | clarkb: sdague: maurosr: i'm going to cancel the pending stable checks running for https://review.openstack.org/53940 and then recheck in about 10 minutes so it'll run on available nodes | 19:44 |
fungi | oh, wait, i can't cancel those via jenkins since they're not running | 19:45 |
fungi | maybe once gearman reports the difference to zuul they'll pick up automagically? | 19:45 |
fungi | guess we'll know here shortly | 19:46 |
fungi | yep--they're running already. magic! | 19:48 |
*** mestery has joined #openstack-infra | 19:54 | |
*** rakhmerov has joined #openstack-infra | 19:54 | |
*** ericw has quit IRC | 19:55 | |
openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Move base test case logic out of __init__.py https://review.openstack.org/48860 | 20:04 |
*** marun has joined #openstack-infra | 20:05 | |
lifeless | fungi: IIRC you're organising a keysigning in HK ? | 20:07 |
lifeless | fungi: a) when [please don't say 'evening'] and b) have you announced it on d-d ? | 20:07 |
fungi | lifeless: no, what i said was we want to organize an official ksp at the j summit | 20:08 |
lifeless | fungi: oh | 20:08 |
lifeless | fungi: cause I have this shiny new 4K key | 20:08 |
lifeless | fungi: and I wants sigs :> | 20:08 |
fungi | lifeless: i'll have my passport and cards with my key fingerprint on it. still (at this very moment in fact) drafting up the beginnings of recommendations for our community (drawing mostly from an amalgam of ubuntu/debian/suse/et al) | 20:10 |
lifeless | fungi: cool | 20:10 |
lifeless | fungi: I will get you and zigo_ and thats enough to have it in the web of trust to make it official in Debian | 20:11 |
fungi | want something more succinct that a laundry lists of links to other communities recommendations, for obvious reasons | 20:11 |
*** reed has joined #openstack-infra | 20:11 | |
fungi | wow, that was some horrible typing on my part | 20:11 |
fungi | something more succinct THAN a laundry list of linke | 20:11 |
fungi | links | 20:11 |
* fungi gives up typing | 20:11 | |
fungi | hoping i'll have a message to the -dev ml this evening (my time) with a link to the initial wiki article | 20:12 |
fungi | in hk i'm primarily focused on getting strong signatures between all the core members of the release cycle management program, but will encourage ad hoc signing between anyone who wants to do so | 20:13 |
lifeless | if you have a small ksp for that group, count me in | 20:14 |
fungi | and then at the j summit, the goal is to hopefully get string wot ties between the rcm cores and anyone signing official things (announcements, tarballs, whatev) | 20:15 |
fungi | you bet | 20:15 |
lifeless | I was thoroughly in the scc some years back :) | 20:15 |
fungi | er, strong wot... i seriously need to get used to this new keyboard | 20:15 |
lifeless | fungi: wearable? | 20:17 |
fungi | lifeless: not going to try to get my wearable gear through customs into hk. i'll have it (or its successor) with me at the next one in the usa | 20:18 |
lifeless | fungi: I mean what you're using now? | 20:19 |
anteaya | mordred: keep in mind that seafile configuration generates two secret key/ids which I have no idea how to do | 20:19 |
fungi | oh, a ducky channel mechanical at my standing desk. cherry mx blue switches with #50 cushions on the stems to reduce travel and noise | 20:19 |
anteaya | fungi had said on Friday that we would feed them in via hiera or something and I don't know how to do that | 20:20 |
lifeless | anteaya: hiera is a k:v store within puppet, stored on the puppet master and queryable by puppet rules | 20:20 |
anteaya | right | 20:21 |
fungi | anteaya: you make them variables in the template for that file and then we fill those variables in the global site.pp file | 20:21 |
anteaya | how | 20:21 |
anteaya | how do we get the right values? | 20:21 |
anteaya | I don't know how to do that | 20:21 |
lifeless | anteaya: thats done by hand following the seafiles installation instructions | 20:21 |
fungi | anteaya: you could install it and then tell us what initial values it put in the files if the means by which it generates them is so opaque that we can't figure it out straight away | 20:22 |
lifeless | anteaya: and then copied into hiera | 20:22 |
anteaya | the _entire_ reason I am doing this with seafile and not using owncloud is to avoid doing anything by hand | 20:22 |
anteaya | if I am allowed to do anything by hand, I vote owncloud, it is way simplier | 20:22 |
anteaya | for user and for me | 20:22 |
fungi | anteaya: we want it to be repeatable without doing anything by hand, but presumably something gets installed by hand once on a dev system to reverse-engineer the configuration and stiff | 20:23 |
fungi | stuff | 20:23 |
lifeless | anteaya: these secrets, are they per-user, or per-install ? | 20:23 |
anteaya | lifeless per install | 20:23 |
lifeless | anteaya: ok, so putting secrets into hiera doesn't count as 'by hand' the same way 'apt-get install' or other actions do. | 20:23 |
lifeless | anteaya: strange but true | 20:24 |
fungi | from earlier discussions, it's sounded like both owncloud and seafile generate initial admin accounts when the first user connects to the webui and store the initial creds in files as a one-way hash | 20:24 |
anteaya | but the only way I know of to get the values to stick into hiera is to execute the install script | 20:24 |
anteaya | fungi: mostly correct | 20:24 |
anteaya | that is true for owncloud | 20:25 |
anteaya | for seafile the user runs an interactive script on the server | 20:25 |
lifeless | anteaya: ok so you would do the generation somewhere secure, copy the files and gpg encrypt them to e.g. fungi, and he can decrypt and put in hiera | 20:25 |
fungi | anteaya: but if you execute the install script on a dev system once and figure those values out, then we can put them in puppet so subsequent installations just get configuration applied by puppet | 20:25 |
fungi | right, what lifeless said | 20:25 |
anteaya | great | 20:25 |
anteaya | so for seafile installation, I can assume that I can ssh to the server and run the generation script? | 20:26 |
fungi | the trick comes when we want to change what those values are in the future, which involves either changing them through the application in question and copying the results or reverse-engineering the code which stores those values so we can perform the same transformations on our own | 20:27 |
*** rakhmerov has quit IRC | 20:27 | |
anteaya | fungi yes, but do we expect that we need to change the RSA key for that server to use seafile | 20:28 |
anteaya | or the uuid for one of the databases | 20:28 |
fungi | anteaya: ssh into your dev system, run the installation script, copy off the configuration files that generates and put them in puppet. files with sensitive data in them, turn into templates, replace the sensitive bits with variable substitutions (which we'll replace through hiera) and tell us what they were | 20:28 |
anteaya | since I can put the other stuff in the puppet templates | 20:28 |
anteaya | okay | 20:28 |
lifeless | anteaya: if the secrets are meant to be unique per installation | 20:29 |
lifeless | anteaya: then the other option is to just encode running the setup script on each server as it's brought up | 20:29 |
fungi | if we need to replace keys or passwords later, we can work out a pragmatic means of creating new ones (either through the application or by reverse-engineering as i mentioned) | 20:29 |
lifeless | anteaya: one would do that if each server (even serving the same hostname) should have unique creds | 20:30 |
anteaya | lifeless: I would love to do that, any examples of how I can encode running the setup script? | 20:30 |
fungi | running an interactive setup script through stdin redirection or expect-like automation tends to be a bit fragile, so if we can just use/template the configs directly that would be preferable | 20:30 |
anteaya | okay | 20:31 |
anteaya | can I get some direction on what the root directory of the app should be? | 20:31 |
lifeless | anteaya: there are plenty, but we'd need to see if this situation qualifies | 20:31 |
anteaya | the manual has it in home/user | 20:31 |
anteaya | less than ideal by my estimation | 20:31 |
anteaya | lifeless: okay thanks | 20:32 |
fungi | anteaya: probably something in /srv like maybe /srv/seafile or /srv/owncloud if it's going to contain a lot of data. putting it under /usr/local is maybe better if it's mostly executables/libs and not really data. /opt is also a potential but, meh, feels like the 90s called and they want their mountpoint back | 20:33 |
anteaya | ha ha ha | 20:34 |
*** marun has quit IRC | 20:34 | |
anteaya | me thinks it will run heavier to data than executables | 20:34 |
anteaya | into /srv/seafile it goes | 20:34 |
fungi | that sounds good. we do most of our web content on other servers in /srv similarly | 20:35 |
anteaya | k | 20:36 |
anteaya | now for my vm I need to set up dbs using mysql, but I know we are going to use trove so I won't puppet the creation of the dbs | 20:36 |
anteaya | it asks for a db user, db pass, dp ip and db name | 20:37 |
anteaya | so that tells me we can use trove, correct? | 20:37 |
fungi | yep | 20:37 |
*** UtahDave has joined #openstack-infra | 20:37 | |
anteaya | great | 20:37 |
fungi | use a local mysql service to mock that for now, and then we'll change it in the config to use hiera variables which we'll fill with the trove connection/auth details | 20:38 |
fungi | so on your dev system it's probably 127.0.0.1 or localhost for the db ip, but we'll change that in production to use the trove instance dns name instead | 20:38 |
fungi | and we'll similarly generate db auth credentials which we'll set in trove and in hiera | 20:39 |
*** markmcclain has joined #openstack-infra | 20:40 | |
*** dcramer_ has quit IRC | 20:40 | |
*** markwash has joined #openstack-infra | 20:43 | |
*** dims has joined #openstack-infra | 20:44 | |
*** senk has joined #openstack-infra | 20:45 | |
anteaya | okay | 20:51 |
*** chuck_ is now known as zul | 20:52 | |
*** markmcclain has quit IRC | 20:53 | |
*** Bada has joined #openstack-infra | 20:53 | |
sdague | clarkb: yeh, so I think summarize was a missing piece of my understanding of graphite | 21:05 |
sdague | that being said, having to define the counters in advance, and having to do everything via url edit, makes me kind of sad | 21:05 |
*** ArxCruz has joined #openstack-infra | 21:06 | |
sdague | http://graphite.openstack.org/graphlot/?from=00:00_20130701&until=23:59_20131027&target=summarize(sumSeries(stats_counts.zuul.pipeline.check.all_jobs),%20'1d') - is that just because the meter is new? or do we only keep a couple months of data in graphite? | 21:07 |
anteaya | mordred: you want a modules/seafile/manifests/site.pp file, as well as or instead of manifests/site.pp? | 21:08 |
clarkb | sdague: I think it is new, before then we didn't distinguish on pipeline maybe | 21:09 |
clarkb | sdague: these are the sorts of things that graphite isn't great at, its much more rigid | 21:10 |
sdague | yeh | 21:10 |
clarkb | anteaya: you should only have a modules/seafile/manifests/site.pp if you need it | 21:10 |
sdague | it would be nice if there were gerrit counters for official programs, incubated, and stackforge | 21:10 |
clarkb | manifests/site.pp is where we will configure host(s) to use seafile | 21:10 |
*** nati_ueno has joined #openstack-infra | 21:11 | |
anteaya | okay, mordred wants me to have that | 21:11 |
anteaya | so I will nuke what I have in manifests/site.pp at the moment | 21:11 |
clarkb | anteaya: you need to edit manifests/site.pp for sure | 21:11 |
anteaya | I do | 21:11 |
clarkb | yes that is the file where you tell puppet what to do with individual nodes that call home | 21:11 |
anteaya | so I need both then | 21:11 |
sdague | http://graphite.openstack.org/graphlot/?from=00:00_20130401&until=23:59_20131027&target=summarize(sumSeries(stats_counts.gerrit.event.change-merged),%20'1d') is cool (total merges per day) | 21:11 |
clarkb | anteaya: maybe, depends on what you need modules/seafile/manifests/site.pp for :) | 21:12 |
anteaya | to edit manifests/site.pp and to create modules/seafile/manifests/site.pp | 21:12 |
sdague | but hard to slice it | 21:12 |
anteaya | clarkb: yeah, I think mordred wants me to have the same info in it that I current have in manifests/site.pp | 21:12 |
clarkb | ah | 21:12 |
anteaya | but I am unsure why | 21:13 |
anteaya | I'll do both for now | 21:13 |
anteaya | deleting is easy | 21:13 |
clarkb | anteaya: your changes to manifests/site.pp look fine to me | 21:13 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 21:13 |
anteaya | clarkb: thanks | 21:14 |
anteaya | :D | 21:14 |
clarkb | anteaya: the contents of modules/seafile/manifests/site.pp don't do anything though | 21:14 |
clarkb | anteaya: the reason mordred wants a modules/seafile/manifests/site.pp is probably due to https://review.openstack.org/#/c/54030/8/modules/openstack_project/manifests/seafile.pp line 31 | 21:15 |
clarkb | that line attempts to include a thing that would be in modules/seafile/manifests/site.pp | 21:15 |
anteaya | oh | 21:15 |
anteaya | I just copied what I saw from other modules | 21:16 |
anteaya | 19 different variables for this configuration | 21:16 |
clarkb | anteaya: right, some modules use modules/thing/manifests/site.pp to do site specific configuration and modules/thing/manifests/init.pp to do installation | 21:16 |
anteaya | I am brewing a talk on creating auto configurable apps | 21:17 |
anteaya | okay | 21:17 |
clarkb | I don't think you can auto config everything in all but the most trivial cases, but people writing software do need to stop pretending that anyone wants to isntall and configure their software by hand | 21:17 |
sdague | clarkb: so where did we move devstack launches to in graphite? | 21:18 |
sdague | http://graphite.openstack.org/graphlot/?width=586&height=308&_salt=1382908610.338&from=00%3A00_20130401&until=23%3A59_20131027&lineMode=staircase&connectedLimit=&areaMode=first&drawNullAsZero=true&minXStep=&xFormat=%25m%2F%25d&target=summarize(sumSeries(stats_counts.devstack.launch.*)%2C%20'1d')&target=summarize(sumSeries(stats_counts.devstack.job.*.builds)%2C%20'1d')&target=summarize(sumSeries(stats_counts.zuul.job.*devstack*)%2C%20'1d') | 21:18 |
sdague | all of those end suspiciously around the same time | 21:18 |
clarkb | sdague: this is why I start with zuul's graphs, let me look | 21:18 |
clarkb | stats_counts.zuul.pipeline.*.all_jobs is what zuul summarizes, I think it is in that * | 21:19 |
clarkb | er no | 21:19 |
clarkb | it should be something like stats_counts.zuul.pipeline.*.*-tempest-devstack-* | 21:19 |
clarkb | hmm no data | 21:20 |
sdague | yeh, I actually want *devstack* to get the largeops and swift jobs | 21:21 |
clarkb | sdague: stats_counts.zuul.pipeline.*.job.*-tempest-devstack-*.* | 21:21 |
*** senk has quit IRC | 21:22 | |
clarkb | first splat is for the various queues, second splat is for gate/check third splat for the variations on devstack deployments and last splat is for job result, failure/success/abort | 21:22 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 21:23 |
*** senk has joined #openstack-infra | 21:23 | |
sdague | ok, it seems to be "thinking" | 21:23 |
*** rakhmerov has joined #openstack-infra | 21:24 | |
*** DennyZhang has joined #openstack-infra | 21:26 | |
sdague | any idea if there is an easy way to do series division? | 21:27 |
fungi | sdague: as far as official/incubated/other categories, we also need somewhere to track which programs are in what state and what projects they contain. that's hopefully going to live in a registry within openstack/governance which should hopefully make it a little easier to start auto-categorizing project-specific data along those lines once there is a single source of truth for it | 21:28 |
clarkb | sdague: http://graphite.readthedocs.org/en/1.0/functions.html#graphite.render.functions.divideSeries ? | 21:28 |
fungi | right now we have several places where we guess at the state of various projects, which has a tendency to diverge and go stale | 21:29 |
fungi | so (i think at least subconsciously) we're avoiding making more of those places | 21:29 |
fungi | clarkb: looking at the test nodes graph, do you get the feeling that we're accumulating cruft/ghost nodes in a deleting state? | 21:30 |
clarkb | fungi: yes, for rax I think | 21:31 |
fungi | grr | 21:31 |
clarkb | fungi: I don't think nodepool is able to successfully bring them up so they go into the delete pool and we just spin there | 21:31 |
fungi | oh, that makes slightly more sense at least. i wonder what happened with the rax images | 21:31 |
clarkb | either boot timeouts or the image is bad | 21:32 |
fungi | hopefully i'll have a moment to start looking once i wrap up my cleaning frenzy here | 21:33 |
fungi | (a tidy laboratory is an efficient laboratory!) | 21:33 |
sdague | ok, that seemed to work - http://graphite.openstack.org/graphlot/?from=00:00_20130728&until=23:59_20131027&target=divideSeries(summarize(sumSeries(stats_counts.zuul.pipeline.gate.job.*devstack*.FAILURE),%20'1d'),%20summarize(sumSeries(stats_counts.zuul.pipeline.gate.job.*devstack*.*),%20'1d')) | 21:36 |
sdague | that's the devstack fail percentage in the gate | 21:36 |
sdague | though from a user point of view, the zuul reset rate might be more interesting. Guess we need counters on that. | 21:37 |
*** juice has quit IRC | 21:39 | |
*** juice has joined #openstack-infra | 21:39 | |
fungi | that's something we might be able to infer based on the pipeline (as least as an upper bound) but definitely need to hear it explicitly from zuul for any real accuracy | 21:39 |
clarkb | any failure in the gate pipeline is a reset | 21:41 |
clarkb | with a * next to that saying its one per change | 21:41 |
clarkb | which I don't think graphite captures | 21:41 |
fungi | and a ** next to it reminding you that there are multiple shared queues in the gate pipeline, so which one are you tracking | 21:43 |
*** senk has quit IRC | 21:44 | |
*** rakhmerov has quit IRC | 21:57 | |
sdague | is there a way to get graphite to dump the data sets out in json or something for local graphing? | 21:58 |
clarkb | sdague: there is | 21:59 |
clarkb | sdague: &format=json | 21:59 |
sdague | cool | 22:01 |
sdague | man, their date format is crazy pants HH:MM_yyyymmdd | 22:04 |
sdague | so I take it the "login" in graphite to save graphs on our instance is a myth? | 22:05 |
*** dcramer_ has joined #openstack-infra | 22:08 | |
*** DennyZha` has joined #openstack-infra | 22:09 | |
*** DennyZhang has quit IRC | 22:12 | |
clarkb | sdague: I think you might be able to save them somehow, but jeblair has deleted that data in the past | 22:13 |
sdague | heh | 22:15 |
mordred | fungi: I can has +2/+A here https://review.openstack.org/#/c/52776/ ? | 22:20 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 22:22 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 22:25 |
openstackgerrit | A change was merged to openstack-infra/gerritlib: Remove use of d2to1 https://review.openstack.org/52776 | 22:26 |
fungi | mordred: is gerritlib getting added to the projects list in openstack/requirements? | 22:26 |
fungi | (so as to receive future magic requirements update patches? | 22:26 |
openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate current clients on stable branches https://review.openstack.org/41931 | 22:31 |
mordred | fungi: I do not believe that would be appropriate, as it is not required to only use those moving forward | 22:32 |
fungi | mordred: oops--in that case the comment line you added to gerritlib's setup.py in 52776 was wrong | 22:35 |
mordred | fungi: nod. there you go | 22:37 |
openstackgerrit | Monty Taylor proposed a change to openstack-infra/gerritlib: Remove erroneous comment https://review.openstack.org/54037 | 22:37 |
*** Bada has quit IRC | 22:38 | |
openstackgerrit | A change was merged to openstack-infra/gerritlib: Remove erroneous comment https://review.openstack.org/54037 | 22:38 |
openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use pip install instead of setup.py install https://review.openstack.org/51004 | 22:39 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Create a seafile instance. https://review.openstack.org/54030 | 22:46 |
*** fbo is now known as fbo_away | 22:50 | |
*** rakhmerov has joined #openstack-infra | 22:54 | |
anteaya | pleia2: do you know where is an example in the puppet files of downloading and unpacking a tarball? | 22:56 |
openstackgerrit | A change was merged to openstack-dev/pbr: Prevent ordereddict installation on 2.7 https://review.openstack.org/52152 | 22:58 |
*** rakhmerov has quit IRC | 22:59 | |
anteaya | I can find how to use git to clone a repo or set up a ppa | 22:59 |
anteaya | but so far, no examples of downloading a tarball | 22:59 |
pleia2 | anteaya: grepping through the repo the only one I can find is drupal + modules/openstack_project/manifests/groups_dev.pp | 23:06 |
pleia2 | but it's not using puppet to do it, it's calling a script | 23:07 |
pleia2 | not sure how we actually use this | 23:07 |
*** senk has joined #openstack-infra | 23:09 | |
anteaya | thanks pleia2, I'll take a look | 23:09 |
anteaya | then probably going to pack it in for the night | 23:10 |
anteaya | I haven't had the time to set up puppet on a vm and take this patch for a drive | 23:10 |
anteaya | so review or not as you please | 23:11 |
*** saper has quit IRC | 23:19 | |
*** blamar has quit IRC | 23:20 | |
anteaya | yup, my brain is done | 23:23 |
anteaya | I'll continue tomorrow | 23:23 |
anteaya | night | 23:23 |
*** anteaya has quit IRC | 23:23 | |
*** saper has joined #openstack-infra | 23:25 | |
*** DennyZha` has quit IRC | 23:31 | |
*** blamar has joined #openstack-infra | 23:32 | |
*** michchap_ has quit IRC | 23:37 | |
*** michchap has joined #openstack-infra | 23:39 | |
*** oubiwann-lambda has quit IRC | 23:42 | |
*** DennyZha` has joined #openstack-infra | 23:52 | |
*** rakhmerov has joined #openstack-infra | 23:55 | |
*** DennyZha` has quit IRC | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!