Monday, 2014-07-28

*** bdpayne has quit IRC00:00
*** harlowja_at_home has joined #openstack-infra00:02
*** gokrokve has quit IRC00:03
*** homeless has quit IRC00:04
lifelessmordred: so https://review.openstack.org/#/c/94108/ is passing CI00:06
*** dkranz has quit IRC00:09
*** yaguang has joined #openstack-infra00:13
*** _nadya_ has quit IRC00:26
*** yamamoto has quit IRC00:32
*** unicell has joined #openstack-infra00:33
*** yaguang has quit IRC00:46
*** julim has quit IRC00:50
*** _nadya_ has joined #openstack-infra00:52
*** pcrews has joined #openstack-infra01:04
*** zz_dimtruck has joined #openstack-infra01:09
*** zz_dimtruck is now known as dimtruck01:09
*** dimtruck is now known as zz_dimtruck01:12
*** yaguang has joined #openstack-infra01:13
*** harlowja_at_home has quit IRC01:16
*** r1chardj0n3s is now known as r1chardj0n3s_afk01:18
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Use correct provider in test-case  https://review.openstack.org/10918401:22
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Filter negative available count for allocator  https://review.openstack.org/10918501:22
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Revert "Revert "Track last allocations to ensure forward-progress""  https://review.openstack.org/10989001:22
*** harlowja_at_home has joined #openstack-infra01:22
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Change history allocation from list to dict  https://review.openstack.org/10918701:27
*** yamamoto_ has joined #openstack-infra01:35
*** banix has joined #openstack-infra01:37
*** nosnos has joined #openstack-infra01:43
*** xianghui has joined #openstack-infra01:47
*** pcrews has quit IRC01:50
*** dims has quit IRC01:52
*** dims has joined #openstack-infra01:54
*** ramashri_ has quit IRC01:56
*** ramashri has joined #openstack-infra02:00
*** harlowja_at_home has quit IRC02:02
*** ramashri has quit IRC02:02
*** ramashri has joined #openstack-infra02:12
*** dims has quit IRC02:14
*** amotoki has joined #openstack-infra02:19
*** otter768 has quit IRC02:30
*** yamahata_ has joined #openstack-infra02:37
*** bradm has quit IRC02:46
*** enikanorov_ has quit IRC02:56
*** enikanorov__ has joined #openstack-infra02:56
*** KanagarajM has joined #openstack-infra03:04
*** homeless has joined #openstack-infra03:06
*** amalagon has joined #openstack-infra03:11
*** homeless has quit IRC03:11
*** otter768 has joined #openstack-infra03:13
*** oomichi has joined #openstack-infra03:16
*** lttrl has quit IRC03:23
*** oomichi has quit IRC03:23
*** oomichi has joined #openstack-infra03:23
openstackgerritKanagaraj Manickam proposed a change to openstack-infra/config: namos - openstack infra device and service manager  https://review.openstack.org/10949403:30
*** HenryG is now known as HenryG_afk03:31
*** nosnos has quit IRC03:37
*** amalagon_ has joined #openstack-infra03:39
*** shashankhegde has joined #openstack-infra03:40
*** amalagon has quit IRC03:42
*** jcoufal has joined #openstack-infra03:44
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Work around for single hiera data dir  https://review.openstack.org/8422903:52
*** dims has joined #openstack-infra03:54
*** chandankumar has joined #openstack-infra03:55
*** yamamoto_ has quit IRC03:58
*** daya_k has joined #openstack-infra03:59
*** yamamoto_ has joined #openstack-infra03:59
*** daya_k has quit IRC03:59
*** yamamoto_ has quit IRC04:04
*** yfried has quit IRC04:12
*** nosnos has joined #openstack-infra04:15
*** xingchao has joined #openstack-infra04:15
*** otter768 has quit IRC04:27
*** xingchao has quit IRC04:32
*** banix has quit IRC04:33
*** bradm has joined #openstack-infra04:42
*** jcoufal has quit IRC04:47
*** ildikov has quit IRC04:56
*** yaguang_ has joined #openstack-infra04:57
*** xianghuihui has joined #openstack-infra04:57
*** shashankhegde has quit IRC04:58
*** Guest79468 is now known as gpocentek04:58
*** gpocentek has joined #openstack-infra04:58
*** yaguang has quit IRC04:58
*** xianghui has quit IRC04:59
*** dims has quit IRC05:00
*** yaguang__ has joined #openstack-infra05:05
*** xianghuihuihui has joined #openstack-infra05:05
openstackgerritGauvain Pocentek proposed a change to openstack/requirements: Update openstack-doc-tools to at least 0.17  https://review.openstack.org/10990505:06
*** yaguang_ has quit IRC05:07
*** xianghuihui has quit IRC05:07
*** yfried has joined #openstack-infra05:11
*** jamielennox is now known as jamielennox|away05:11
*** stevemar has quit IRC05:11
*** k4n0 has joined #openstack-infra05:17
*** fabo has quit IRC05:23
*** fabo has joined #openstack-infra05:23
*** yamamoto_ has joined #openstack-infra05:23
openstackgerritA change was merged to stackforge/python-jenkins: make JSON depth response configurable  https://review.openstack.org/9310505:28
openstackgerritIan Wienand proposed a change to openstack-infra/config: Initial RHEL7/Centos7 support  https://review.openstack.org/10345805:29
openstackgerritIan Wienand proposed a change to openstack-infra/config: Add centos7 image for rax  https://review.openstack.org/10990605:29
*** e0ne has joined #openstack-infra05:43
*** e0ne has quit IRC05:48
*** e0ne has joined #openstack-infra05:52
*** e0ne has quit IRC05:54
*** hogepodge has quit IRC05:55
*** unicell has quit IRC05:57
*** e0ne has joined #openstack-infra05:58
*** lukego has joined #openstack-infra06:00
*** zz_dimtruck is now known as dimtruck06:00
*** unicell has joined #openstack-infra06:01
*** e0ne has quit IRC06:03
*** hogepodge has joined #openstack-infra06:04
*** yfried_ has joined #openstack-infra06:04
*** yfried has quit IRC06:05
*** e0ne has joined #openstack-infra06:06
*** hogepodge has quit IRC06:07
*** e0ne has quit IRC06:07
*** zehicle_at_dell has quit IRC06:08
*** dimtruck is now known as zz_dimtruck06:10
*** yfried__ has joined #openstack-infra06:10
*** e0ne has joined #openstack-infra06:12
*** yfried__ is now known as yfried06:13
*** yfried_ has quit IRC06:13
*** e0ne has quit IRC06:17
*** ildikov has joined #openstack-infra06:18
*** oomichi has quit IRC06:19
*** oomichi has joined #openstack-infra06:19
*** e0ne has joined #openstack-infra06:22
*** e0ne has quit IRC06:23
openstackgerritRadomir Dopieralski proposed a change to openstack/requirements: Add xstatic-packaged JavaScript libraries for Horizon  https://review.openstack.org/9779306:24
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604306:27
*** e0ne has joined #openstack-infra06:27
*** e0ne has quit IRC06:30
*** afazekas is now known as __afazekas06:33
*** e0ne has joined #openstack-infra06:33
*** cnesa5 has joined #openstack-infra06:34
*** tchaypo1 is now known as tchaypo06:35
*** KanagarajM has quit IRC06:36
*** oomichi has quit IRC06:37
*** afazekas_ has quit IRC06:39
*** e0ne has quit IRC06:39
*** andreykurilin has joined #openstack-infra06:43
*** dims has joined #openstack-infra06:45
*** KanagarajM has joined #openstack-infra06:45
*** dims has quit IRC06:49
*** flaper87 has quit IRC06:57
*** flaper87 has joined #openstack-infra06:57
*** _nadya_ has quit IRC07:03
*** mkerrin has joined #openstack-infra07:04
*** cnesa6 has joined #openstack-infra07:14
*** _nadya_ has joined #openstack-infra07:17
*** cnesa5 has quit IRC07:17
*** afazekas_ has joined #openstack-infra07:18
*** shardy has quit IRC07:23
*** ihrachyshka has joined #openstack-infra07:28
*** e0ne has joined #openstack-infra07:28
*** ihrachyshka_ has joined #openstack-infra07:32
*** ihrachyshka has quit IRC07:33
*** luqas has joined #openstack-infra07:35
*** _nadya_ has quit IRC07:35
*** luqas has quit IRC07:36
*** isviridov|vacati is now known as isviridov07:37
*** luqas has joined #openstack-infra07:37
*** yolanda has joined #openstack-infra07:38
*** hashar has joined #openstack-infra07:39
*** jlibosva has joined #openstack-infra07:39
*** _nadya_ has joined #openstack-infra07:40
*** IvanBerezovskiy has joined #openstack-infra07:42
*** jistr has joined #openstack-infra07:42
*** Longgeek has joined #openstack-infra07:44
*** jedimike has joined #openstack-infra07:48
*** Clabbe has joined #openstack-infra07:50
*** lpaduano has joined #openstack-infra07:52
*** jpich has joined #openstack-infra07:53
*** andreykurilin has quit IRC07:55
*** ramashri has quit IRC07:55
*** _nadya_ has quit IRC07:55
*** andreykurilin has joined #openstack-infra07:55
*** lpaduano has left #openstack-infra07:56
*** pelix has joined #openstack-infra07:56
*** ramashri has joined #openstack-infra07:58
*** e0ne has quit IRC07:59
*** zz_dimtruck is now known as dimtruck07:59
*** jlibosva has quit IRC08:04
*** jlibosva has joined #openstack-infra08:05
*** berendt has joined #openstack-infra08:06
*** yaguang_ has joined #openstack-infra08:06
*** xianghuihui has joined #openstack-infra08:07
*** _nadya_ has joined #openstack-infra08:07
*** xianghuihuihui has quit IRC08:08
*** yaguang__ has quit IRC08:08
*** dimtruck is now known as zz_dimtruck08:09
*** mrmartin has joined #openstack-infra08:10
*** dguerri is now known as _dguerri08:12
*** pblaho has joined #openstack-infra08:18
*** _dguerri is now known as dguerri08:18
*** _nadya_ has quit IRC08:18
*** dtantsur|afk is now known as dtantsur08:19
*** derekh_ has joined #openstack-infra08:21
*** andreykurilin has quit IRC08:21
*** e0ne has joined #openstack-infra08:22
*** jlibosva has quit IRC08:25
*** jlibosva has joined #openstack-infra08:25
*** Guest17840 is now known as BobBall08:30
*** chmouel has quit IRC08:30
*** chmouel has joined #openstack-infra08:32
*** salv-orlando has joined #openstack-infra08:32
openstackgerritDina Belova proposed a change to openstack-infra/config: New python-opentsdbclient repo  https://review.openstack.org/10863008:34
*** joachim is now known as joa08:37
Nglifeless: you .o/'d last night. I was snoozing. sup?08:38
joahi o/08:38
*** rkukura_ has joined #openstack-infra08:47
*** rlandy has joined #openstack-infra08:48
*** rkukura has quit IRC08:50
*** ZZelle has joined #openstack-infra08:51
*** e0ne has quit IRC08:52
*** rkukura_ has quit IRC08:54
*** skolekonov has joined #openstack-infra08:57
*** ociuhandu has joined #openstack-infra08:59
*** luqas has quit IRC08:59
*** _nadya_ has joined #openstack-infra09:04
*** niska has quit IRC09:08
*** yamamoto_ has quit IRC09:10
*** e0ne has joined #openstack-infra09:14
*** niska has joined #openstack-infra09:15
*** che-arne has quit IRC09:31
*** luqas has joined #openstack-infra09:37
*** che-arne has joined #openstack-infra09:39
*** berendt has quit IRC09:39
*** yaguang_ has quit IRC09:44
*** xianghuihui has quit IRC09:46
izinovikhi stackers09:48
izinovikis it appropriate place to ask questions about how nova-network functions ?09:48
kashyapizinovik, This channel is for  OpenStack infrastructure related topics09:49
izinovikkashyap, could you redirect me to right place ?09:49
kashyapizinovik, If you're facing issues w/ current git, then #openstack-nova is the right place, if you're facing issues with current stable release (IceHouse), then probably ask.openstack.org09:50
*** e0ne has quit IRC09:55
*** e0ne has joined #openstack-infra09:55
*** pcm_ has joined #openstack-infra09:56
*** pcm_ has quit IRC09:57
*** pcm_ has joined #openstack-infra09:58
*** openstackgerrit has quit IRC10:01
*** jcoufal has joined #openstack-infra10:05
*** _nadya_ has quit IRC10:13
*** amalagon has joined #openstack-infra10:16
*** amalagon_ has quit IRC10:19
*** hashar_ has joined #openstack-infra10:20
*** hashar has quit IRC10:23
*** tteggel_ is now known as tteggel10:25
*** dtantsur is now known as dtantsur|lunch10:25
*** unicell has quit IRC10:28
yolandahi anteaya, i just updated publication with right formatting, and updated content10:29
*** hashar_ has quit IRC10:35
joaanteaya: krtaylor_: I don't see much changes compared to when I left for my holidays on the Next 3rd-party meeting agenda. Was the meeting cancelled ? I should be available for today's meeting (maybe not up until the end but at least partly)10:35
*** jp_at_hp has joined #openstack-infra10:37
*** unicell has joined #openstack-infra10:37
*** che-arne has quit IRC10:38
*** yamahata_ has quit IRC10:38
*** yamamoto has joined #openstack-infra10:39
*** _nadya_ has joined #openstack-infra10:50
*** dguerri is now known as _dguerri11:02
*** yfried_ has joined #openstack-infra11:10
*** yfried has quit IRC11:10
*** yfried__ has joined #openstack-infra11:11
*** luqas has quit IRC11:12
*** luqas has joined #openstack-infra11:14
*** yfried_ has quit IRC11:14
*** luqas has quit IRC11:15
*** luqas has joined #openstack-infra11:17
*** afazekas_ is now known as afazekas11:21
*** yfried__ is now known as yfried11:25
*** jcoufal has quit IRC11:25
*** dosaboy_ is now known as dosaboy11:25
*** KanagarajM has quit IRC11:31
*** KanagarajM has joined #openstack-infra11:33
*** dtantsur|lunch is now known as dtantsur11:36
*** HenryG_afk is now known as HenryG11:39
*** yamamoto has quit IRC11:39
*** e0ne has quit IRC11:39
*** e0ne has joined #openstack-infra11:46
*** KanagarajM has quit IRC11:48
*** amotoki has quit IRC11:50
*** miqui has quit IRC11:54
*** krtaylor_ is now known as krtaylor11:56
*** yamamoto has joined #openstack-infra11:57
krtaylorjoa, no, see the meeting logs, we had a good discussion and cinder ci status overview, we saved your topic to discuss with you11:59
*** matrohon has joined #openstack-infra11:59
*** dmsimard_away is now known as dmsimard12:05
anteayayolanda: okay great12:05
anteayahi joa and krtaylor12:05
_dguerriHello there. anteaya told me that she want to discuss this review https://review.openstack.org/109292 (inclusion of git-upstream in stackforge) during the next infra meeting. Who can add this topic to the agenda, please?12:05
anteayakrtaylor: so do you have time to talk about december?12:05
anteaya_dguerri: you can12:05
*** dkranz has joined #openstack-infra12:05
_dguerrioh, really... cool :)12:05
_dguerriwill do then12:05
*** _dguerri is now known as dguerri12:05
anteayahttps://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting12:06
krtaylorhi anteaya12:06
anteayahi krtaylor12:06
anteayarunning away to a meeting or do you have a few?12:06
krtayloranteaya, sure, for a little bit, warning - I am caffeine depleted12:06
anteayanp12:06
anteayaso december12:06
anteayawhat things do we want to accomplish?12:07
anteayaone item is a face to face with third party ci admins so they can meet each other12:07
anteayaand teach each other12:07
*** hashar has joined #openstack-infra12:07
anteayatutorial style12:07
krtayloranteaya, its hard for me to think past summit, but yes12:08
anteayaalso clarkb had a good suggestion to open it up for more general automated testing topics12:08
anteayaand draw in other folks12:08
anteayawell we have to get something figured out for there to be an event12:08
anteayaelse it disappears12:08
anteayawhat do you want to accomplish?12:08
krtayloranteaya, I think some way for summaries of systems to be shared by those that cannot attend would be good  define12:08
dguerrianteaya thanks. just added it12:09
anteayadguerri: great thanks12:09
anteayakrtaylor: how is that different from what we have now?12:09
krtayloranteaya,  well, I was thinking it could be joint infra/thirdparty, we'll need infra there to get anything done12:09
anteayait would be good to have as much of infra as can attend12:10
anteayaI disagree we need infra there to get anything done12:10
krtayloranteaya, what we have now could work, the systems pages could be that framework, if it was used that way12:10
anteayaI think the folks attending third party meetings are getting a lot done12:10
anteayalet's work towards using that way12:10
krtaylorabsolutely, not what I meant12:10
anteayaI want infra there, but I don't want the success of the event to be defined by jim's schedule for instance12:11
krtayloralso infra work12:11
anteayasince we haven't asked him what he wants yet12:11
anteayaright12:11
anteayawe can have some code sprint type work done12:11
anteayalike on pulling out a module or somethign12:11
krtayloryes, I want more work on post summit decisions on jenkins, testing, storyboard, etc12:13
krtaylornot just third party focused12:13
anteayasounds good12:13
anteayaI don't know about storyboard12:13
krtaylorlike just happened12:13
anteayathat would be tough to pull off12:13
anteayasounds like you want an infra meetup12:13
krtaylorwhys that12:13
*** dtantsur has quit IRC12:14
krtayloryes, with ci improvement as the focus12:14
krtaylorand other things, but primarily CI12:14
krtaylorI think the meetup/sharing event needs to happen at summit12:15
*** sandywalsh has joined #openstack-infra12:15
krtaylorit will be hard for someone with a limited travel budget to attend both12:15
anteayaI think we are not at all on the same page12:15
*** lttrl has joined #openstack-infra12:15
anteayawhat just took place in darmstadt was a qa/infra meetup12:15
anteayathat is not what I want this event to be12:16
anteayaI want this event to be a chance for third party people to meet and learn from other thrid party people12:16
anteayasince there seems to be some myth than they can only do things once someone has blessed them12:16
anteayaand I need to dispell that myth12:16
anteayathere are many things third party folks can do to self regulate12:17
anteayaand teach each other12:17
anteayaand offer patches to code and documents that would improve their experience12:17
*** ArxCruz has joined #openstack-infra12:17
krtaylorabsolutely, thats why I wanted to form this group12:17
anteayathat is what I would like to focus on12:17
anteayathat is what I would like us to discuss12:17
krtaylorbut I don't think it would work as a mid-cycle12:18
*** yamamoto has quit IRC12:18
anteayait isn't a mid-cycle12:18
anteayathis is a third party meetup12:18
*** jcoufal has joined #openstack-infra12:18
anteayaexpanded to include other folks interested in automated testing and quality frameworks12:18
anteayasome of which may have nothing to do with openstack12:18
krtaylorthen why not at summit with better attendance?12:18
krtayloroh, not OS, hm12:19
anteayaopenstack is openstack12:19
anteayaOS is operating system12:19
krtaylorheh, with more coffee it is12:19
anteayabecause I have no time at summit12:19
anteayaand noone else in infra has time at summit12:19
anteayaand summit is already planned12:19
anteayawe can't plan an event to take place at summit now12:19
anteayawe have no space in the schedule and no rooms12:20
anteayasummit planning took place long ago12:20
anteayawe can't just add events to summit12:20
krtaylorok, thats why I was wondering12:20
anteayagood thing we are talking12:20
*** mbacchi has joined #openstack-infra12:21
krtaylorI doubt all sessions are planned, but I agree that space has been allocated12:21
anteayaso we need an event for third party folks to meet and talk to and learn from other third party and automated testing folks12:21
anteayathe time for sessions has been allocated12:21
krtaylorlast summit, I was trying to organize a meetup where we used some common area, it turned out to be the dev lounge, so it wouldnt have worked anyway12:22
anteayathe most you can do at summit is gather some folks for drinks, informally12:22
anteayasince formal events are by lottery under the organization of the foundation12:22
krtayloranteaya, ok, I am beginning to understand your vision, and I agree it is needed, what I saw all along too12:23
anteayaso let's consider summit unavailable for this kind of event12:23
anteayagreat12:23
krtaylorI just see some problems getting good attendance12:23
anteayaso Austin12:23
anteayait depends on what we are offering12:23
anteayaand I'm curious to know what you mean by good attendance12:23
krtayloranteaya, I have sent out the request through the proper channels, should know soon12:23
anteayaI see 30 - 50 being no problem at all12:24
anteayakrtaylor: okay thanks, keep me informed12:24
*** nelsnelson has joined #openstack-infra12:24
krtaylorreally, my last event, we didnt break 2012:24
anteayasince it if takes too long or the answer is no, we have to come up with a back up plan12:24
anteayakrtaylor: which event was that?12:24
krtayloralthough it was tight focused, and planned last minute12:24
anteayaI think the planned last minute might be the key there12:25
krtayloranteaya, agreed, it might not be a bad idea to see about a second location as a backup12:25
anteayaokay I put out some feelers already and can follow up with an email12:25
* krtaylor notices his empty coffee cup12:26
anteayawhat event was the last event to which you are refering?12:26
*** pdmars has joined #openstack-infra12:27
krtaylorI organized a Linux/Android multimedia mini-summit in Austin on our campus, I was the Linaro multimedia tech lead at that time12:27
*** pdmars has quit IRC12:27
anteayathat isn't even the same audience12:27
krtayloron assignment into Linaro from IBM12:27
krtayloranteaya, nope, not even close12:28
anteayaso yay for getting 20 people12:28
anteayawell done12:28
*** pdmars has joined #openstack-infra12:28
yolandaanteaya, thx, i think formatting is fine now, but i'd like to have some feedback about topics12:28
anteayaso since I have zero plans to set up an entity to collect admission fees, let's just agree to make the event fee-less12:28
anteayawe get someone to pay for the room at a hotel and we ensure the hotel is close to restaurants12:29
* krtaylor isn't sure but that he may have just been insulted on his summit marketing skillz :)12:29
anteayaand then attendees do their own rooms and food and we get someone to cover the cost of the meeting room and av needs12:29
anteayakrtaylor: not at all12:29
anteayaI was complimenting you12:30
*** rfolco has joined #openstack-infra12:30
krtayloranteaya, agreed, no registration12:30
anteayano we need registration12:30
krtayloranteaya, it would be IBM sponsored12:30
anteayawhat we don't need is fees12:30
krtaylorright no fee12:30
anteayawell good12:30
krtaylorthey would have to pay to get there and lodging12:30
anteayalet's get an email saying IBM will cover the room at a hotel and av needs12:30
anteayathen we will acknowledge them as sponsor12:30
krtaylorno, on campus12:30
anteayahmmmmm12:31
krtaylorthats the only way we could do it12:31
anteayareally?12:31
krtayloryeah, room/av is "free" that way12:31
anteayawell what kinds of food options are available on IBM campus?12:31
krtaylorwe have full catering from our on-site cafeteria12:32
anteayaby attendees will have to pay our ot pocket12:32
krtaylorand restaurants close12:32
anteayawhat choices will attendees have12:32
anteayahow close for restaurants?12:32
anteayacan people walk in 10 minutes or less?12:32
krtayloranteaya, I'd see about IBM picking that cost up, maybe dinners are on the group but breakfast/lunch shouldnt be a problem12:33
krtayloranteaya, there are several close, 10-15 min walk12:33
anteayawell if IBM wants to pay for two meals a day, that is acceptable to have it from the campus caterers then12:34
anteayakrtaylor: how close are hotels?12:34
krtaylorI can't say for sure IBM will yet or not, but I can ask :)12:34
anteayathat is fine12:34
anteayajust outlining what we would need12:34
krtaylorthere are prob 4-5 hotels within a mile12:34
*** chandankumar has quit IRC12:34
anteayaokay12:34
anteayawhat size of rooms can we get?12:35
*** aysyd has joined #openstack-infra12:35
krtayloranteaya, like I said, I have a notebook with all we'll need to get done, all contacts, everything12:35
anteayagreat12:35
*** weshay has joined #openstack-infra12:35
anteayawhat size of rooms are available?12:35
krtayloranteaya, room - I think I can get one of the big rooms, 40-50 shouldnt' be a problem12:35
krtaylorwe have 3 or 4 that would work12:36
anteayais there a room that would take 75 people if we got that many?12:36
krtaylorno, I think we'd have to cap it, but that seems to be a common approach12:36
anteayaso the largest room on the ibm campus could take 50 people max?12:37
krtayloranteaya,  I need to check that12:38
anteayakk12:38
krtaylormaybe larger12:38
anteayalet's find out12:38
*** eharney has joined #openstack-infra12:39
anteayaso let's ensure we get confirmation that we can get a meeting room for at least 50 people for the first two days of Dec 1-5 2014, and break out rooms for the last 3 days of that week, about 15, 20 people per room, 3 or 4 of those12:39
anteayaav needs12:39
anteayaand breakfast and lunch for 50 people12:39
anteayacan you get that confirmation by August 15th?12:40
anteayabecause if you can't, I will need time to find an alternate12:40
*** adalbas has joined #openstack-infra12:40
*** luqas has quit IRC12:41
krtayloranteaya, will do my best, anything else?  :)12:43
anteayakrtaylor: that is all I can think of12:43
anteayaif we can get confirmation of that by August 15th, then we have enough to move ahead12:43
anteayaif other requirements come up, we can address them as they arise12:43
anteayabut we need this in place to even say we are having an event12:43
anteayaand thanks12:43
anteayakeep me posted12:44
krtaylorthe other prob is the time, right after turkey day in the US I believe12:44
* anteaya moves off to breakfast and resetting the pressure switch on the pump12:44
krtaylorsome might not be back from holidy to fly out on the 1st12:44
anteayakrtaylor: then more space for folks from not us to attend12:45
anteayaI think we will be full with a waiting list12:45
anteayamy estimate12:45
krtayloranteaya, we can talk about that part later though12:45
krtaylorcool12:45
krtaylorI was thinking 3-4 day would work too12:45
krtaylorbut let me find the rooms and get approval, then we can figure that out12:46
*** dguerri is now known as _dguerri12:46
*** _dguerri is now known as dguerri12:46
*** dguerri is now known as _dguerri12:46
*** _dguerri is now known as dguerri12:46
*** maxbit has joined #openstack-infra12:47
*** vhoward has joined #openstack-infra12:49
*** k4n0 has quit IRC12:50
*** che-arne has joined #openstack-infra12:52
*** dprince has joined #openstack-infra12:54
*** dguerri is now known as dguerri`afk12:54
*** dguerri`afk is now known as dguerri12:54
*** jgallard has joined #openstack-infra12:56
*** yamamoto has joined #openstack-infra12:56
*** mriedem has joined #openstack-infra12:59
*** yamamoto has quit IRC13:01
yolandahere, i have a question. I need to upload a change, but it depends on another change in different project13:01
yolandahow does this work?13:02
*** dkliban has joined #openstack-infra13:03
*** dkranz has quit IRC13:05
*** ihrachyshka_ has quit IRC13:07
*** nosnos has quit IRC13:07
*** luqas has joined #openstack-infra13:09
*** prad has joined #openstack-infra13:12
*** AJaeger has joined #openstack-infra13:12
anteayayolanda: you have to manually state the dependency in a comment in the patch13:13
anteayaand tests will fail until the dependency is merged, if tests depend on it13:14
* anteaya hopes she has finished any home plumbing for the day13:14
yolandaanteaya, so i leave a message in comment, something like Depends-On: xxx ?13:14
anteayawell I wouldn't put in the commit message13:15
anteayajust as a comment on the patch13:15
*** julim has joined #openstack-infra13:15
anteayabut yes, something like that13:15
*** eharney has quit IRC13:15
yolandaok13:15
yolandaanteaya, i'm working on some changes to allow having project descriptions automated into Gerrit, so i need updates in gerritlib and jeepyb, that's why i asked13:15
anteayamakes sense13:16
anteayaand awesome work13:16
*** dkranz has joined #openstack-infra13:17
*** bknudson has quit IRC13:18
AJaegerSergeyLukjanov, fungi : Could you review https://review.openstack.org/109795, please?13:18
*** yamamoto has joined #openstack-infra13:27
*** cnesa6 has quit IRC13:27
joaanteaya: Now that I've unstacked my mails, back to CI (Now to set-up the cinder specific part) \o/13:28
*** eharney has joined #openstack-infra13:28
anteayajoa: \o/13:28
anteayaand welcome back13:28
*** banix has joined #openstack-infra13:29
joaI hope my etherpad feedback before leaving was useful to you13:29
*** cnesa6 has joined #openstack-infra13:29
anteayaI honestly can't remember13:30
*** otherwiseguy has joined #openstack-infra13:30
anteayadon't take that personally though13:30
anteayado you have the etherpad link?13:30
joano worries. I still have it on hand13:31
anteayaor remember the topic?13:31
anteayagreat13:31
joahttps://etherpad.openstack.org/p/Setting_up_a_3rd-party_CI___Issues_encountered13:31
anteayathanks13:31
* anteaya reads again13:31
joawas kind of a summary of everything I had to understand before being able to set up the basic CI13:31
anteayaoh yes, yes yes yes13:31
anteayathis is awesome13:31
anteayathank you13:31
joaThought that making a list out of it might be a good basis for a tutorial-like thing13:31
*** yamamoto has quit IRC13:32
*** ihrachyshka has joined #openstack-infra13:32
anteayaright13:32
anteayagood idea13:32
anteayawe have a new repo: http://git.openstack.org/cgit/openstack-infra/infra-manual/13:32
anteayait is a manual13:32
joaoh good13:32
anteayaand spot for third party has just been created13:32
anteayawould you be willing to transfer your etherpad to the manual?13:32
joaI guess it will be updated properly in parallel of the main openstack-infra/config repo ?13:33
*** openstackgerrit has joined #openstack-infra13:33
joaI am,13:33
anteayahere is where it is published: http://docs.openstack.org/infra/manual/13:33
anteayano, the manual is independent of the config repo13:33
anteayaand we are working on making the infra-manual to be a one stop guide for folks13:34
joamhhh I mean, if some important thing changes (like the syntax to match trigger comments), it should be reflected in the manual repo too13:34
anteayaso if there is material published in two places we are working on making the material availabe in the manual and deleting other sources13:34
anteayaoh yes13:34
*** pcrews has joined #openstack-infra13:34
anteayayes, I see what you mean, yes, the manual must be kept up to date13:35
openstackgerritA change was merged to openstack-infra/storyboard: Project Group Delete  https://review.openstack.org/10644113:35
joaOkay so the third-party is still empty as I see13:35
joado you want me to putmy etherpad in a... redacted way ? (or at least a first attempt)13:35
anteayastart with one section13:35
anteayalook at the developer guide for an example13:36
anteayabreak it up with headings13:36
anteayasince it will be the initial patch part of the feedback might be on how to set up this section13:36
anteayaso have a thick skin13:36
anteayaoffer something and we will go from there13:37
joaokay, might  not be done for today but I'll work on it. Do you have any deadline for this ?13:37
anteayano13:37
anteayano deadline13:37
*** lukego has quit IRC13:37
joaalright13:37
anteayajust offer something when you are ready, don't try to make it a masterpiece, just offer some patch13:38
anteayaand we will review13:38
joaokay13:38
anteayathanks13:38
anteaya:D13:38
* anteaya is afk13:38
*** bknudson has joined #openstack-infra13:38
joayou're welcome (also, if my feedback can help other people this way, it's all the better)13:38
openstackgerritA change was merged to openstack-infra/storyboard: Test database using migrations  https://review.openstack.org/10702213:39
*** stevemar has joined #openstack-infra13:39
openstackgerritA change was merged to openstack-infra/storyboard: Team REST API added  https://review.openstack.org/10513313:40
*** ihrachyshka has quit IRC13:41
*** ihrachyshka has joined #openstack-infra13:41
openstackgerrityolanda.robla proposed a change to openstack-infra/gerritlib: Add functionality to Gerritlib to set project descriptions  https://review.openstack.org/10995713:43
*** rkukura has joined #openstack-infra13:44
*** signed8bit has joined #openstack-infra13:44
*** ihrachyshka has quit IRC13:45
*** Shrews_ is now known as Shrews13:46
*** mriedem has quit IRC13:47
openstackgerrityolanda.robla proposed a change to openstack-infra/gerritlib: Add functionality to Gerritlib to set project descriptions  https://review.openstack.org/10995713:47
openstackgerritRadomir Dopieralski proposed a change to openstack-infra/config: Create a repository for additional Tuskar-UI plugins  https://review.openstack.org/10810213:50
*** maxbit_ has joined #openstack-infra13:50
*** ihrachyshka has joined #openstack-infra13:50
*** homeless has joined #openstack-infra13:50
*** salv-orlando_ has joined #openstack-infra13:51
*** salv-orlando has quit IRC13:52
*** salv-orlando_ is now known as salv-orlando13:52
*** maxbit_ has quit IRC13:52
*** maxbit_ has joined #openstack-infra13:52
*** julim has quit IRC13:52
*** maxbit_ has quit IRC13:53
*** xianghuihui has joined #openstack-infra13:53
*** julim has joined #openstack-infra13:55
*** maxbit has quit IRC13:55
*** funzo has quit IRC13:55
*** dims has joined #openstack-infra13:56
*** maxbit has joined #openstack-infra13:56
*** UtahDave has joined #openstack-infra13:57
*** UtahDave has left #openstack-infra13:57
*** ihrachyshka has quit IRC13:57
*** ihrachyshka has joined #openstack-infra13:57
*** flaviof_zzz is now known as flaviof13:57
*** salv-orlando has quit IRC13:58
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Session states now use Notifications  https://review.openstack.org/10397913:59
*** salv-orlando has joined #openstack-infra14:00
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Fixed notifications.  https://review.openstack.org/10851314:01
*** timrc-afk is now known as timrc14:02
*** amotoki has joined #openstack-infra14:02
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Added user icon to storyboard icon variables.  https://review.openstack.org/10915814:03
SergeyLukjanovAJaeger /me looking14:04
AJaegerSergeyLukjanov, thanks!14:04
AJaegerand fungi: Thanks as well!14:04
SergeyLukjanovAJaeger, approved14:04
*** ihrachyshka has quit IRC14:05
*** dustins has joined #openstack-infra14:05
AJaegerSergeyLukjanov, there's one more I'd like to see in for translations: https://review.openstack.org/108464 - and then there's the specs publishing: https://review.openstack.org/106100 . Thanks!14:05
*** ihrachyshka has joined #openstack-infra14:06
openstackgerritDoug Hellmann proposed a change to openstack-infra/config: Add test for oslo-cookiecutter repository  https://review.openstack.org/11000514:07
openstackgerritDoug Hellmann proposed a change to openstack-infra/config: Add gate job for jjb changes  https://review.openstack.org/11000614:07
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604314:07
*** tristanC_ is now known as tristanC14:10
*** dguerri is now known as dguerri`afk14:10
*** zul has joined #openstack-infra14:12
*** juzuluag has joined #openstack-infra14:13
openstackgerritA change was merged to openstack-infra/config: Do not add obsolete translation entries  https://review.openstack.org/10979514:13
*** dguerri`afk is now known as dguerri14:13
*** funzo has joined #openstack-infra14:14
*** msabramo has quit IRC14:15
*** jgrimm has joined #openstack-infra14:15
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Point search requests to search endpoints  https://review.openstack.org/10253614:17
*** blitzrage is now known as leifmadsen14:17
*** garyh has joined #openstack-infra14:18
*** yfried has quit IRC14:22
*** jistr has quit IRC14:22
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Adding Search endpoints and sqlalchemy impl  https://review.openstack.org/10147614:23
*** jistr has joined #openstack-infra14:23
openstackgerritA change was merged to openstack-infra/elastic-recheck: add message strings to recheckwatchbot.yaml  https://review.openstack.org/10928014:23
*** rainya has joined #openstack-infra14:24
*** rainya has quit IRC14:24
*** rainya has joined #openstack-infra14:25
openstackgerrityolanda.robla proposed a change to openstack-infra/gerritlib: Add functionality to Gerritlib to set project descriptions  https://review.openstack.org/10995714:26
*** zz_dimtruck is now known as dimtruck14:27
*** yamamoto has joined #openstack-infra14:27
jbrycehi infra folks, what’s the total number of projects running through our infrastructure now? (i know i’ve asked this before, but i couldn’t easily lay my hands on the link to list)14:28
*** david-lyle has joined #openstack-infra14:28
*** yfried_ has joined #openstack-infra14:29
mordredjbryce: 37014:30
mordredjbryce: and just fyi, the following command will tell you that ... ssh -p 29418 review.openstack.org gerrit ls-projects | wc -l14:30
rcarrillocruzi assume that contains stackforge ?14:30
*** mfink has joined #openstack-infra14:31
mordredyup. 172 if you grep out stackforge14:31
jbrycemordred: thanks. can you point me to where the file is that contains the list?14:31
mordredjbryce: absolutely - one sec14:31
mordredjbryce: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/review.projects.yaml14:31
*** mfink has quit IRC14:31
*** yamamoto has quit IRC14:32
jbryceawesome thanks14:32
jbrycegathering data for irs meeting tomorrow morning14:32
mordredooh. excellent14:32
mordredthe stackforge modules then are actually pretty important then14:33
mordredas many of them don't have direct things to do with openstack14:33
*** baohua has joined #openstack-infra14:33
*** ihrachyshka has quit IRC14:34
*** juzuluag has quit IRC14:34
jbrycemordred: exactly14:35
*** ihrachyshka has joined #openstack-infra14:36
*** mfink has joined #openstack-infra14:37
*** Longgeek_ has joined #openstack-infra14:38
openstackgerritA change was merged to openstack-infra/config: add libvirtd logs to logstash  https://review.openstack.org/10814114:38
*** needscoffee has joined #openstack-infra14:38
needscoffeegood morning :)14:39
openstackgerrityolanda.robla proposed a change to openstack-infra/jeepyb: Allow create/update projects with description into Gerrit  https://review.openstack.org/11000914:40
openstackgerritA change was merged to openstack-infra/infra-manual: Remove duplicated words  https://review.openstack.org/10955514:40
*** Longgeek has quit IRC14:40
*** wenlock has joined #openstack-infra14:43
*** betsy has quit IRC14:43
*** ihrachyshka has quit IRC14:43
openstackgerrityolanda.robla proposed a change to openstack-infra/jeepyb: Allow create/update projects with description into Gerrit  https://review.openstack.org/11000914:44
*** Longgeek_ has quit IRC14:44
*** atiwari has joined #openstack-infra14:44
*** ihrachyshka has joined #openstack-infra14:45
*** cdub has quit IRC14:46
*** Longgeek has joined #openstack-infra14:48
*** hogepodge has joined #openstack-infra14:50
*** thedodd has joined #openstack-infra14:52
krtaylorsweston, ping, were you going to create a terms patch for third-party.rst?14:53
krtaylorsweston, else I can if you want14:53
*** mrda-traveling is now known as mrda14:55
*** zul has quit IRC14:56
*** ihrachyshka_ has joined #openstack-infra14:56
*** ildikov has quit IRC14:56
*** Longgeek has quit IRC14:57
yolandahi mordred, i sent you a review for the openstack-infra/config nodepool stuff14:57
*** datsun180b has joined #openstack-infra14:57
yolandait was related to linux-headers-* packages for devstack, they aren't working for debian14:57
yolandai was thinking in removing the package install from the element, and move it to devstack_node script, as we do with other dependent packages14:58
yolandawhat do you think?14:58
*** zns has joined #openstack-infra14:59
*** ihrachyshka has quit IRC14:59
krotscheckstoryboard meeting in #openstack-meeting-315:00
*** salv-orlando has quit IRC15:00
*** dprince has quit IRC15:00
openstackgerritA change was merged to openstack-infra/storyboard: Added LIKE search to project groups  https://review.openstack.org/10645715:00
openstackgerritA change was merged to openstack-infra/storyboard: Project Group member DELETE and PUT method have better results  https://review.openstack.org/10650715:00
dhellmannmordred: if you have a sec, I need another +2 on the change to add oslo alphas to the global requirements list15:01
dhellmannmordred: https://review.openstack.org/#/c/109338/15:01
fungijbryce: if you need any more detailed data, let me know15:02
*** salv-orlando has joined #openstack-infra15:03
*** baohua has quit IRC15:03
yolandamordred, jeblair, i also have a working version of nodepool , quite tested, can i get some review on it?15:03
mordreddhellmann: zomg. does it actually work now?15:04
dhellmannmordred: as far as I can tell15:04
mordredyolanda: yah. seems fine by me (the move of linux-headers)15:04
mordredyolanda: and yes, I'll look at the patches today15:04
mordreddhellmann: woot!15:04
fungimordred: at least until the cheeseshop breaks it (currently under discussion)15:04
yolandamordred, thx15:04
dhellmannmordred: that pbr fix for argparse was the last change, and that works locally, so...15:04
mordredfungi: wait - what?15:04
yolandai can do the changes of linux-headers myself15:05
fungimordred: did you see the thread on the packaging sig list last week?15:05
dhellmannfungi: shoot me15:05
mordredfungi: no. I did not15:05
yolandamordred, i'll just update your openstack-infra/config patch with that15:05
*** maxbit has quit IRC15:05
mordredfungi: is there a tl;dr or should I go read the thread?15:05
mordredyolanda: awesome15:05
fungiconcerns have been raised that allowing uploads of wheels without accompanying tarballs could lead to pypi distributing compiled binaries without distributing source for them, which might violate some free software licenses15:05
yolandai also pushed a pair of changes to gerritlib and jeepyb that can be useful15:06
* fungi will dig up the thread15:06
AJaegerhttps://review.openstack.org/#/c/106100 and https://review.openstack.org/#/c/109675/ now have two +2s by fungi and SergeyLukjanov (thanks!) but no approval. Any infra core around for the +A, please?15:06
mordredfungi: the same can be done right now with eggs15:07
mordredwhy is this pypi's problem?15:07
fungimordred: yeah, i know. i think there are just people who feel they need to be the license police15:07
mordredoh, well, they REALLY don't want to15:07
mordredbecause then they open themselves to liability, and it's almost impossible to get right15:08
mordredthey really want ot be a common carrier15:08
mordredand behave as one15:08
fungimordred: dhellmann: https://mail.python.org/pipermail/distutils-sig/2014-July/024575.html15:08
fungir1chardj0n3s_afk: can probably discuss at length when he's awake15:09
mordredr1chardj0n3s_afk: I would VERY much like to talk to you about that ^^^15:09
fungidstufft also had some input on that thread (basically pointed out that we're doing exactly that right now)15:09
mordredawesome15:09
* dhellmann considers building a pypi replacement15:10
mordred"Also, reject uploads that are not released under a DFSG license or lack man15:10
mordredpages."15:10
mordredplease tell me that daniel holth was trolling15:10
fungii mainly only injected a plea for an alternate means of accomplishing what we're doing now via some more sane mechanism if they're seriously entertaining making that enforcement choice15:10
mordredthat's the most absurd thing I've ever heard15:10
openstackgerritDaniel Shirley proposed a change to openstack-infra/jenkins-job-builder: Added support for JClouds Blobstore  https://review.openstack.org/10751515:10
*** amotoki has quit IRC15:10
dstufftmordred: it's trolling15:10
fungiyeah, i think the dfsg comment was trolling15:11
dstufftdaniel has an odd sense of hjumor15:11
fungior more to the point, illustrative sarcasm15:11
mordredok15:11
mordredjust making sure15:11
mordredit's tough to tell with dfsg comments ... :)15:12
dstufftI'm meh on mandating sdists15:12
salv-orlandomestery, mtreinish, sdague: we’re reading to make the neutron full job voting. Overall failure rate below 10%, below 5% if patches with problems are excluded. if you have time, please re-review: https://review.openstack.org/#/c/88289/15:12
mordredNick says sane things15:12
* salv-orlando obvisouly wanted to write “we’re ready"15:12
jbrycefungi: the yaml gives me most of what i need. always interested in use cases of the non-integrated projects if anyone’s heard of new ones15:12
dstufftI understand the reasoning behind it, but I don't think it's going to be overlly useful and it prevents a trick15:12
mordred"Packages without readily available15:12
mordredsource uploads just won't be redistributed"15:12
dstufftyea15:12
*** dkehnx has quit IRC15:12
dstufftbasically that15:12
mordreddstufft: what if we _do_ have a source tarball for a different version?15:13
dstufftthere are lots of ways to make it so a linux can't distribute your thing15:13
*** zns_ has joined #openstack-infra15:13
fungijbryce: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml is probably also a useful accompaniment (for the non-stackforge projects anyway)15:13
mordredI can't believe I'm about to say the next thing ...15:13
dstufftmordred: I dunno, I wasn't at EuroPython15:13
dstufftI know about as much as is on that thread15:14
mordreddstufft: they do realize that there are other things that are important than just getting included in the distros, right?15:14
dstufftI think it's a more or less pointless argument made by some random distro people15:14
mordred++15:14
dstufftbut that's OK, I'm getting better at arguing with distro people15:14
*** zns has quit IRC15:14
mesterysalv-orlando: thanks for the udpate and for the work on this!15:15
*** dkehn_ has joined #openstack-infra15:15
dstufftmordred: just to be clear, I doubt anything will come of it15:15
mordreddstufft: phew15:15
* fungi can't imagine any distro worth discussing would be retrieving a wheel and building a distro package based on its contents15:15
* mordred was considering subscribing to the list to join in15:15
*** dkehn_ is now known as dkehnx15:15
mordredfungi: also, it turns out you don't have to distribute source with binaries, even for the GPL15:15
mordredthe source just has to be available15:16
dstufftfungi: well I think the thing is that by not uploading a sdist a distro can't package your thing15:16
dstufftand the distros obviously don't like that15:16
fungimordred: correct. i suspect the "concern" raised is that wheels (which as you point out is also just as true for eggs) could encourage upstreams to no longer bother distributing source code corresponding to their releases15:16
*** dangers_away is now known as dangers15:16
dhellmannit's true that not having the sdist on pypi makes distro packaging slightly more challenging for smaller projects15:16
mordredyup15:17
mordredbut you know what, the packager who wants to package a project should discuss that with the upstream if/when the packager wants to package that project15:17
mordredit's not a thing that's systemically accomplished15:17
dhellmannyeah15:17
mordredin debian, an ITP is filed, generally by a human who cares15:17
fungii think the problem is the other way 'round though... if you don't put your release tarballs somewhere easily accessible to the public at large, distros aren't going to package and redistribute it. not sure why this is a problem for the distros themselves, just for the upstreams15:18
mordredthat human has the ability to find where the source code actually lives15:18
mordredexactly15:18
dstufftfungi: because it makes more work for the distro people if they want to package your thing15:18
dstufftbecause now they have to convince you to publish an sdist15:18
dstufftinstead of being able to just consume an already existing one15:18
*** juzuluag has joined #openstack-infra15:18
mordreddstufft: yup. but that's still not pypi's problem15:19
mordredanywho - dhellmann, I'll go poke your change now15:19
dstufftwell yea, my response is more or less http://i.minus.com/ibflqegIrPmuwL.gif15:19
dhellmannmordred: thanks15:19
mordreddstufft: ++15:19
*** mrmartin has quit IRC15:20
*** otherwiseguy has quit IRC15:21
*** xianghuihui has quit IRC15:21
openstackgerritMatthew Treinish proposed a change to openstack-infra/elastic-recheck: Fix the coverage job in tox  https://review.openstack.org/10978915:22
*** ramashri has quit IRC15:22
*** signed8bit has quit IRC15:22
anteayajbryce: also this patch and its dependency are additions to the governance/programs.yaml file that haven't been merged yet: https://review.openstack.org/#/c/107395/15:23
clarkbbtw I restarted zuul on saturday15:23
jbryceanteaya, fungi: thanks15:24
AJaegerclarkb, thanks!15:24
clarkbthere was a change that git wedged and it was writing gigs of logs. I couldnt figure a better way to unwedge it. was going to debug more this morning15:24
AJaegerbtw. there're three jobs at the top of the check queue that need to be restarted somehow...15:24
anteayajbryce: may those consuming the slides you prepare understand the contents15:25
fungiclarkb: saw that--do you need a bug opened and were you working on a patch?15:25
fungiclarkb: though i guess debugging more first makes perfect sense15:25
*** signed8bit has joined #openstack-infra15:26
clarkbno bug yet or patch15:26
jbryceanteaya: ha! there’s definitely some “up-leveling” i’m working on here = )15:26
anteayamay it go well15:26
anteaya:D15:26
clarkbI wanted to resume weekend :)15:27
fungiunderstandable15:27
fungiand thanks for triaging'15:27
*** yamamoto has joined #openstack-infra15:27
fungiAJaeger: yeah, i suspect the nova-docker change at the top is killing its worker for that one job in such a way that zuul thinks jenkins is broken and keeps restarting it15:28
*** zns_ has quit IRC15:28
fungiAJaeger: the stuck jobs for the nova and neutron changes after it look like zuul never got the completion message from jenkins15:29
*** dims has quit IRC15:29
*** james_li has joined #openstack-infra15:30
fungithe three stuck jobs are all on jenkins05, which isn't responding on its webui at the moment15:30
fungii'll dig deeper15:30
fungialso the gate is full of breaking15:31
fungithat unstable ceilometer docs job seems to have failed to upload some files15:31
AJaegerfungi, wasn't jenkins05 not reachable last week ?15:31
fungistatic.o.o might be in trouble15:31
*** yamamoto has quit IRC15:32
fungiAJaeger: yes, i believe so. this time it's less impacting, so i may leave it broken and open a support ticket15:32
*** yfried_ has quit IRC15:33
fungijenkins05 is responding to icmp echo requests, but i don't get the sshd banner on 22/tcp15:33
*** vermarah has joined #openstack-infra15:35
fungioh yeah, the docs-draft docroot on static.o.o is smack full15:35
AJaegerdocs-draft? Is the documentation team creating too much content? ;(15:36
fungii've added the remaining ~100gb we had in the vg to docs-draft (bringing it up to 1.5tb)15:37
fungias soon as the fs resize completes, i'll start runnign a find to make sure we're properly expiring old content15:38
fungibut at least this should lift the pain15:38
*** esker has joined #openstack-infra15:38
*** david-ly_ has joined #openstack-infra15:40
*** gyee has joined #openstack-infra15:40
fungiyeah, looks like it's just been steady growth... http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=718&rra_id=all15:40
AJaegerfungi, yeah, looks like no deletion or too late15:41
*** otherwiseguy has joined #openstack-infra15:41
*** alexpilotti has joined #openstack-infra15:41
fungiAJaeger: well, i don't think it's that we're not deleting. i think it's just that we're adding more projects with more gate-foo-docs jobs which all publish to docs-drafts and we may need to further shorten our retention window15:42
*** david-ly_ is now known as david-lyle_15:42
*** ihrachyshka_ has quit IRC15:42
*** gangil has joined #openstack-infra15:42
*** david-lyle has quit IRC15:42
openstackgerritNicholas Randon proposed a change to openstack-infra/config: Add firewall traffic on port 27410 to 27510.  https://review.openstack.org/10873715:43
AJaegerfungi, are these that large? I think openstack-manuals is the largest one...15:43
fungii believe when this came up a month ago i pointed out that the cron job which expires old content wasn't being aggressive enough, and SergeyLukjanov said he'd look at patching it15:43
AJaegerThere'S a single dent in the last graph a month ago but steady growth afterwards15:44
*** rainya has quit IRC15:45
*** jgallard has quit IRC15:46
*** otherwiseguy has quit IRC15:46
fungiAJaeger: yeah, that dent was when SergeyLukjanov manually deleted some older files15:47
AJaegerah, ok...15:47
*** amotoki has joined #openstack-infra15:49
*** liam__ has joined #openstack-infra15:49
fungiAJaeger: right now http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/log_archive_maintenance.sh currently only deletes *.gz files older than 21 days under /srv/static/docs-draft (and empty directories) but i think we can get rid of the -name \*.gz in there and have it apply to all files besides what's in lost+found15:51
fungii'm testing that now and will push up a review of the same momentarily15:51
*** zul has joined #openstack-infra15:51
AJaegerindeed, that would not delete lots of files - see http://docs-draft.openstack.org/97/109097/2/check/gate-openstack-manuals-tox-doc-publish-checkbuild/b8cfc81/publish-docs/15:52
AJaegerThat should contain many files that are just plain html - or graphics etc.15:52
openstackgerritArx Cruz proposed a change to openstack-infra/config: Adding auth_version option to zuul.conf  https://review.openstack.org/10903015:52
*** david-lyle_ is now known as david-lyle15:52
*** liam__ has quit IRC15:52
AJaegerfungi, if you do what I guess you change, you get my +1 once I'm back from dinner ;) But first cycling home...15:53
clarkbfungi: ok I think I see the bug, will capture that and push a possible fix15:55
fungiclarkb: awesome15:56
clarkbexcept maybe I need to dig through the 28GB of logs to be really sure15:56
fungiick15:56
clarkbI see why we got stuck. I do not know why we tried to double report a change15:57
fungii'd be willing to blame that part on gerrit15:57
*** marcoemorais has joined #openstack-infra15:58
fungithough you *might* find some java stacktrace in its log from when it tried to report the merge in the event stream or as a result of a query from zuul or something15:58
*** marcoemorais has quit IRC15:58
*** johnthetubaguy is now known as zz_johnthetubagu15:59
*** otherwiseguy has joined #openstack-infra16:00
*** alexpilotti has quit IRC16:01
*** rainya has joined #openstack-infra16:01
*** ramashri has joined #openstack-infra16:02
joaanteaya: krtaylor:  I'm going home to do the meeting from there. I should not be too late but just in case, I'm telling you both.16:02
*** needscoffee has quit IRC16:02
*** pblaho has quit IRC16:02
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Expire docs-drafts more aggressively  https://review.openstack.org/11004516:03
anteayajoa: kk16:03
anteayafungi: glad static.o.o waited for you to return before filling up16:03
*** marcoemorais has joined #openstack-infra16:03
*** jergerber has joined #openstack-infra16:04
clarkbfungi: the longer I look at this the more i am confused. zuul is maintaining this state internally and getting otu of sync with itself16:05
clarkbso I think the fix may be more subtle than I thought /me goes back to drawing board16:05
*** shashankhegde has joined #openstack-infra16:07
*** hashar has quit IRC16:09
*** markmcclain has joined #openstack-infra16:09
*** dmsimard is now known as dmsimard_away16:11
*** markmcclain has quit IRC16:12
*** markmcclain has joined #openstack-infra16:12
*** mrmartin has joined #openstack-infra16:16
clarkbfungi: jhesketh jeblair mordred SergeyLukjanov https://bugs.launchpad.net/zuul/+bug/134948616:17
uvirtbotLaunchpad bug 1349486 in zuul "Change already reported error resulting in pipeline processing error loop" [High,Triaged]16:17
*** e0ne has quit IRC16:18
clarkbI think I am going to have to find my log mandoline and dig into where this started16:18
*** salv-orlando has quit IRC16:18
*** ildikov has joined #openstack-infra16:23
*** luqas has quit IRC16:27
*** yamamoto has joined #openstack-infra16:27
*** salv-orlando has joined #openstack-infra16:28
*** prad has quit IRC16:28
*** Kiall_ is now known as Kiall16:29
fungiokay, the jenkins05 network failure recurrence is opened as support ticket 140728-ord-0010200 and i'm leaving the server in a broken state for now in case that assists the support techs in determining the underlying cause16:30
*** yamamoto has quit IRC16:31
clarkb2014-07-26 00:13:36,483 is approximately when zuul wedged itself /me uses grep -B and -A16:32
clarkbfungi: rgr16:32
*** jcoufal has quit IRC16:33
fungii am going to head out and grab lunch, then come back and hopefully get some proper reviews and coding done16:33
clarkbSSHException: Key-exchange timed out waiting for key negotiation bam that may have been the cause16:34
*** melwitt has joined #openstack-infra16:35
*** markmcclain1 has joined #openstack-infra16:36
*** markmcclain has quit IRC16:36
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for oslo rpc unit test race bug 1288899  https://review.openstack.org/10988316:37
uvirtbotLaunchpad bug 1288899 in oslo "Unrelated failure in tests.unit.rpc.test_fake.RpcFakeTestCase.test_multithreaded_resp_routing" [Low,Triaged] https://launchpad.net/bugs/128889916:37
clarkbhttps://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/scheduler.py#n1447  is where zuul had the ssh exception16:37
clarkbso I should be able to write a patch in a bit16:38
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for oslo unit test race bug 1282691  https://review.openstack.org/10986116:38
*** cnesa6 has quit IRC16:38
uvirtbotLaunchpad bug 1282691 in oslo "Probable race in qpid unit tests" [Low,Triaged] https://launchpad.net/bugs/128269116:38
*** _nadya_ has quit IRC16:39
*** shashankhegde has quit IRC16:39
*** dmsimard_away is now known as dmsimard16:44
*** dmsimard is now known as dmsimard_away16:45
*** dmsimard_away is now known as dmsimard16:45
*** cdub has joined #openstack-infra16:46
*** bearhands is now known as comstud16:46
*** zns has joined #openstack-infra16:46
openstackgerritJC Delay proposed a change to openstack-infra/git-review: get_remote_url(): honor any "url.<base>.insteadOf" config setting  https://review.openstack.org/10985116:49
*** ihrachyshka has joined #openstack-infra16:50
*** jaypipes has joined #openstack-infra16:51
*** armax has joined #openstack-infra16:51
*** jistr has quit IRC16:51
*** marzif_ has quit IRC16:52
*** amalagon has quit IRC16:53
*** praneshp has joined #openstack-infra16:53
clarkbreportItem() is actually fairly complicated..16:53
*** dmsimard is now known as dmsimard_away16:54
*** jlibosva has quit IRC16:56
*** esker has quit IRC16:56
*** jedimike has quit IRC16:57
*** derekh_ has quit IRC16:58
joao/ on time16:59
*** jpich has quit IRC16:59
*** markwash has joined #openstack-infra16:59
*** dguerri is now known as dguerri`afk17:00
*** melwitt has quit IRC17:02
asselinclarkb, I got the nodepool nodes to go offline. How to you get them deleted and new ones created?17:02
*** msabramo has joined #openstack-infra17:02
*** Ryan_Lane has joined #openstack-infra17:02
*** vhoward has left #openstack-infra17:02
*** melwitt has joined #openstack-infra17:02
*** prad has joined #openstack-infra17:03
clarkbasselin: nodepool should do that automatically. it will mark and delete nodes that have completed testing17:04
clarkbasselin: then backfill those nodes with new nodes17:04
*** cdub has quit IRC17:04
*** ihrachyshka has quit IRC17:05
*** dustins has quit IRC17:05
*** cdub has joined #openstack-infra17:05
*** otherwiseguy has quit IRC17:05
*** ihrachyshka has joined #openstack-infra17:05
*** ihrachyshka has quit IRC17:06
*** dustins has joined #openstack-infra17:06
*** _nadya_ has joined #openstack-infra17:06
*** ihrachyshka has joined #openstack-infra17:06
dstufftmordred: do you know anyone at puppetlabs by any chance ;P17:06
*** otherwiseguy has joined #openstack-infra17:06
dstufftI know you use puppet17:06
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Admin Project Group UI  https://review.openstack.org/10528717:06
*** annegent_ has joined #openstack-infra17:07
*** cdub has quit IRC17:07
clarkbdstufft: a couple did or do hang out here17:08
clarkbHunner: ^17:08
*** melwitt has quit IRC17:09
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for cinder DBDeadlock bug 1343274  https://review.openstack.org/10973317:09
uvirtbotLaunchpad bug 1343274 in cinder "DB Deadlock creating volume" [High,Confirmed] https://launchpad.net/bugs/134327417:09
*** melwitt1 has joined #openstack-infra17:09
*** craigbr has joined #openstack-infra17:09
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Secondary Submenu  https://review.openstack.org/10915617:09
*** Ajaeger1 has joined #openstack-infra17:09
*** cdub has joined #openstack-infra17:11
*** esker has joined #openstack-infra17:11
craigbranteaya: Hi17:11
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Team Administration UI Stub  https://review.openstack.org/10915717:11
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Fixed modules for project group admin.  https://review.openstack.org/10943817:12
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added user administration interface  https://review.openstack.org/10943917:12
*** james_li has quit IRC17:13
*** rwsu has joined #openstack-infra17:13
*** shashankhegde has joined #openstack-infra17:13
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added profile side menu and submenu  https://review.openstack.org/10944117:13
*** bdpayne has joined #openstack-infra17:13
*** _nadya_ has quit IRC17:14
*** Ajaeger1 has quit IRC17:14
*** Ajaeger1 has joined #openstack-infra17:14
openstackgerritClark Boylan proposed a change to openstack-infra/zuul: Simplify change reporting to handle errors better  https://review.openstack.org/11006617:16
clarkbfungi: jeblair jhesketh SergeyLukjanov mordred ^17:16
*** esker has quit IRC17:16
*** melwitt1 has quit IRC17:16
*** melwitt has joined #openstack-infra17:16
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added pagesize to browse service.  https://review.openstack.org/10937217:16
*** zehicle_at_dell has joined #openstack-infra17:16
*** hogepodge has quit IRC17:17
*** zehicle_at_dell has quit IRC17:17
*** zehicle_at_dell has joined #openstack-infra17:17
mordredclarkb: item.reported = not self._reportItem(item)17:17
mordredthat's just gross - but yeah17:18
clarkbmordred: ya17:18
clarkbI could change that up too, not quite sure why the logic is inverted there, but that is the way it was17:18
mordredwell, because returning 0 is a success return code, even though 0 is false17:19
clarkboh its the return code from ssh17:19
clarkbI get that code now17:19
clarkbit is to early in the morning17:19
*** harlowja_away is now known as harlowja17:20
mordredclarkb: we could change the name from reported to need_to_report - because if the 0 from ssh gets returned, then you no longer need to report the value ?17:20
*** esker has joined #openstack-infra17:20
mordredbut that probably wants to be a follow on patch17:21
mordredsince it probably has knock on effects17:21
clarkbmordred: ya that would make it easier to read17:22
*** afazekas has quit IRC17:22
*** otherwiseguy has quit IRC17:22
*** amotoki has quit IRC17:23
*** markmcclain1 has quit IRC17:25
*** markmcclain has joined #openstack-infra17:25
*** otherwiseguy has joined #openstack-infra17:27
*** yamamoto has joined #openstack-infra17:27
*** banix has quit IRC17:27
*** MarkAtwood has joined #openstack-infra17:29
*** markmcclain1 has joined #openstack-infra17:29
*** markmcclain1 has joined #openstack-infra17:29
*** markmcclain has quit IRC17:29
*** atiwari has quit IRC17:30
*** yamamoto has quit IRC17:31
*** che-arne has quit IRC17:31
*** che-arne has joined #openstack-infra17:32
*** hogepodge has joined #openstack-infra17:32
ArxCruzjhesketh: hey, ping when you're available :)17:33
*** amcrn has joined #openstack-infra17:34
*** atiwari has joined #openstack-infra17:35
Hunnerdstufft: Sup?17:36
armaxis there anyone that could help with these two reviews? https://review.openstack.org/#/c/106901/ and https://review.openstack.org/#/c/106902/17:37
*** markwash has quit IRC17:37
armaxwe’re trying to wrap up with Neutron DVR17:37
*** markwash_ has joined #openstack-infra17:37
*** marcoemorais has quit IRC17:37
armaxand having an experimental job to test DVR with it would get us a long way in shaking out the last remaining bugs17:38
*** marcoemorais has joined #openstack-infra17:38
armaxI hate to nag people here :) I appreciate anyone’s help17:38
*** signed8bit has quit IRC17:38
*** banix has joined #openstack-infra17:38
*** melwitt has quit IRC17:39
*** __afazekas is now known as afazekas17:40
*** thedodd has quit IRC17:41
*** morganfainberg_Z is now known as morganfainberg17:42
*** e0ne has joined #openstack-infra17:43
asselinclarkb, what's the sequence of event to help debug the issue. Zuul tells jenkins to put the node off line. Nodepool detect's the node is offline, then deletes it. nodepool detects it needs to launch new nodes and does so?17:45
openstackgerritArmando Migliaccio proposed a change to openstack-infra/config: Add experimental job Neutron DVR testing on single node  https://review.openstack.org/10690217:45
*** cdub has quit IRC17:46
*** cdub has joined #openstack-infra17:46
openstackgerritArmando Migliaccio proposed a change to openstack-infra/devstack-gate: Adds DVR related env variable to devstack-vm-gate.sh  https://review.openstack.org/10690117:47
clarkbasselin: zuul tells jenkins to run a job and mark the node it runs the job on offline when the job starts, jenkins completes the job and emits an event over the 0mq event publisher saying the job is done, nodepool receives the job complete event and marks the node that ran that job for deletion, nodepool node deleter thread deletes the node17:47
*** MarkAtwood has quit IRC17:49
*** beecee has joined #openstack-infra17:52
notmynameinteresting. the check jobs are fired off when I -2 a patch17:55
clarkbnotmyname: it has been suggested by jogo iirc to change that17:56
*** dmsimard_away is now known as dmsimard17:56
*** daya_k has joined #openstack-infra17:56
*** achanda has joined #openstack-infra17:58
*** e0ne has quit IRC17:58
*** rainya has quit IRC17:59
*** rainya has joined #openstack-infra17:59
*** marcoemorais has quit IRC17:59
*** marcoemorais has joined #openstack-infra17:59
*** yfried_ has joined #openstack-infra17:59
*** annegent_ has quit IRC18:00
*** amalagon has joined #openstack-infra18:00
*** amalagon has quit IRC18:02
*** melwitt has joined #openstack-infra18:02
*** timrc is now known as timrc-afk18:02
*** pblaho has joined #openstack-infra18:05
*** melwitt has quit IRC18:05
*** melwitt has joined #openstack-infra18:05
*** _nadya_ has joined #openstack-infra18:05
*** e0ne has joined #openstack-infra18:07
*** melwitt has quit IRC18:07
*** melwitt has joined #openstack-infra18:07
*** cdub has quit IRC18:08
*** timrc-afk is now known as timrc18:09
*** lukego has joined #openstack-infra18:10
*** signed8bit has joined #openstack-infra18:10
*** dangers is now known as dangers_away18:10
*** alexpilotti has joined #openstack-infra18:11
*** rainya has quit IRC18:11
*** cnesa6 has joined #openstack-infra18:12
anteayacraigbr: hi18:12
jogonotmyname: yup, talked to jhesketh about getting it done18:13
*** achanda has quit IRC18:13
craigbranteaya: In regards to https://review.openstack.org/108630, is the acl file optional? I thought it was required18:13
clarkbcraigbr: the file itself is required, jeepyb knows where to look for that file by default so you don't have to list it in the projects.yaml file in all cases18:14
*** achanda has joined #openstack-infra18:14
craigbrclarkb:  https://review.openstack.org/108630 doesn't add an acl file and there isn't one there already18:15
craigbrwith the right name18:16
anteayacraigbr: correct18:16
*** thedodd has joined #openstack-infra18:17
*** otherwiseguy has quit IRC18:17
craigbranteaya: That wasn't fixed on the second patch set but it still got a +2 so I was wondering if I was confused18:17
anteayacraigbr: now if there were a line in review.projects.yaml headed acl-config: then an acl file wouldn't be required for that patch18:17
craigbranteaya: right. I checked for that18:17
anteayaor if there is an acl file with the project name already in http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/gerrit/acls/stackforge which there isn't18:18
anteayacraigbr: so you are correct, -1 patch needs an acl file18:18
anteayagood eye, sir18:18
craigbranteaya: I will do that. Thanks18:18
anteayalink to an example of an acl file so they can use that as a template18:18
anteayawe need to ensure they know how to get a +1 when we give a -118:18
craigbrGood idea. I didn't do that on the first one18:19
anteayathat is okay18:21
anteayayou are learning18:21
anteayaand doing a good job18:21
anteayakeep it up18:21
craigbrThe check for an acl file seems like a fairly simple check to write for a check job. I'll see if I can add that18:21
*** melwitt1 has joined #openstack-infra18:22
*** rainya has joined #openstack-infra18:22
*** shashankhegde has quit IRC18:22
*** melwitt has quit IRC18:22
ShrewsAre there any known issues going on with our git repos right now?18:23
Shrewsper: http://logs.openstack.org/39/94439/16/check/check-grenade-dsvm-icehouse/bca2fd6/logs/devstack-gate-setup-workspace-new.txt.gz#_2014-07-28_17_12_50_97318:24
fungiclarkb: interestingly, your 110066 reporting fix to the scheduler does not actually collide with my 10496318:24
clarkbShrews: not that I know of. what are you seeing?18:24
Shrewsclarkb: see link18:24
clarkbShrews: are you here in hilsboro?18:24
Shrewsclarkb: no  :(18:24
fungii bet he's nearish to a hillsborough though18:25
clarkbok that job ran on a rax dfw node which is where the zuul mergers are hosted18:25
clarkbShrews: but I Think that failure may be ok in some cases18:26
*** cnesa6 has quit IRC18:26
clarkbShrews: basically git tries to get refs that may not be there for all the projects18:26
fungiShrews: that job checked out the tip of stable/icehouse for tripleo-incubator18:26
clarkbya I think that isn't what made it fail18:26
* Shrews keeps searching for more error msgs....18:27
*** cnesa6 has joined #openstack-infra18:27
clarkbShrews: look in grenade.sh18:27
*** yamamoto has joined #openstack-infra18:27
clarkber the log with that prefix18:27
*** rainya has quit IRC18:27
fungiShrews: since your change is targeting tempest master and that job is testing tempest master changes against stable/icehouse branches, i think that's correct behavior (unless there was a tripleo-incubator stable/icehouse change ahead of it in the gate, but this failure was in the independent check pipeline so there wouldn't have been)18:27
clarkbShrews: http://logs.openstack.org/39/94439/16/check/check-grenade-dsvm-icehouse/bca2fd6/logs/grenade.sh.txt.gz#_2014-07-28_17_41_24_12518:28
Shrewsfungi: ah, thx for the explanation18:28
Shrewsclarkb: still waiting for that one to d/l....18:28
* Shrews needs more b/w18:28
*** achanda has quit IRC18:28
fungiShrews: the git fetch errors are mainly because git doesn't have a way to query remotes for a ref without trying to get them directly, and then throws ugly errors if they're not there18:28
clarkbya it was slow for me at intel18:28
clarkbI did it thorugh my socks proxy though18:29
*** cdub has joined #openstack-infra18:29
*** melwitt has joined #openstack-infra18:30
*** _nadya_ has quit IRC18:31
Shrewsah, no haz taskflow. thx clarkb and fungi18:31
*** melwitt1 has quit IRC18:31
*** yamamoto has quit IRC18:32
*** rainya has joined #openstack-infra18:32
*** MarkAtwood has joined #openstack-infra18:35
*** dims has joined #openstack-infra18:36
*** dims is now known as Guest3382618:36
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Add task_id to Timeline Event info  https://review.openstack.org/10675918:37
*** Guest33826 has quit IRC18:37
*** angdraug has joined #openstack-infra18:37
angdraugI have a q about gerrit: https://review.openstack.org/10790818:38
*** signed8bit has quit IRC18:38
*** dims_ has joined #openstack-infra18:38
angdrauglast comment from gerrit is "The change requires a local merge to resolve."18:38
*** Sukhdev has joined #openstack-infra18:38
angdraughowever when I rebase the change locally, there's no conflict!18:38
angdraugwhat's it complaining about?18:38
*** shashankhegde has joined #openstack-infra18:38
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Adding api events notifier and listener  https://review.openstack.org/10284218:38
angdraugis its failed merge attempt logged anywhere?18:39
*** flaviof is now known as flaviof_zzz18:39
*** melwitt has quit IRC18:39
*** melwitt has joined #openstack-infra18:40
clarkbangdraug: gerrit uses jgit not C git. jgit is different than C git in its conflcit resolution and sometimes this happens18:40
*** dims_ has quit IRC18:40
clarkbangdraug: pushing a manually update will fix it18:40
clarkb* manual update18:41
*** melwitt has quit IRC18:41
*** melwitt1 has joined #openstack-infra18:41
redrobothi, is Sergey Lukjanov around?  RE: https://review.openstack.org/#/c/109398/18:42
fungiangdraug: right, gratuitously rebasing on the tip of the target branch should solve it even if cgit is smart enough to know there are no conflicts18:42
angdraugok thanks. I guess I was right to suspect that it won't carry the +1/+2 from previous patch set in this scenario18:42
*** craigbr has quit IRC18:42
fungiangdraug: it *may* (gerrit relies on comparing the git patch-id which is fairly forgiving)18:42
*** achanda has joined #openstack-infra18:43
angdraugactually yeah, it just did18:43
*** craigbr has joined #openstack-infra18:43
fungii actually think it's *too* forgiving in that regard18:43
fungigit was written for maintaining c source code, where whitespace is non-semantic18:43
*** melwitt1 has quit IRC18:43
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Add commands to go to the prev/next change in the list  https://review.openstack.org/10963718:44
*** melwitt has joined #openstack-infra18:44
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Add a command to return to the change list  https://review.openstack.org/10963618:44
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Allow the default project change list query to be customized  https://review.openstack.org/10963518:44
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Add an example gertty.yaml for OpenStack  https://review.openstack.org/10963418:44
fungiso git patch-id ignores whitespace changes18:44
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Use account table in search  https://review.openstack.org/10963318:44
openstackgerritJames E. Blair proposed a change to stackforge/gertty: Add account table  https://review.openstack.org/10963218:44
clarkbjeblair: ohai! :) when you get a chance there are a couple zuul bug fixes floating around18:44
fungiwhich for python are often quite significant18:44
jeblair(teensy bugfix to the patch at the bottom of that stack)18:44
*** signed8bit has joined #openstack-infra18:44
jeblairclarkb: plural? :)18:44
clarkbjeblair: ya fungi has one and I had one18:44
clarkb11066 and 10496318:45
jeblairclarkb: thx, will review now18:45
clarkbarg I used lp for the zuul bug18:45
clarkbI fail18:45
fungimine is probably fairly low-priority, and just addresses the changes which fail to get reported on when a parent change fails in a dependent pipeline18:45
jeblairclarkb: zuul bugs are still in lp18:45
clarkboh good18:45
fungiwhereas clarkb's addresses a (hopefully rare) deadlock18:45
redrobotSergeyLukjanov ping18:46
jeblairclarkb: what was the issue that triggered your deadlock?18:46
fungi(pipeline deadlock, not a process deadlock thankfully)18:47
clarkbjeblair: late friday, zuul stopped doing all work not in check. It got an SSHException when trying to check if a change merged. At that point it went into an infinite loop failing to report that change18:47
clarkbjeblair: basically every time it processed queues it failed on that change, then tried to process queues again18:47
*** esker has quit IRC18:48
clarkbjeblair: if you go to zuul.o.o:/var/log/zuul/clarkb there is a file with the relevant logs18:48
clarkbI had to slice through many gigs of logs to find that :)18:48
jeblairclarkb: so the trigger was a transient network error at a certain place?18:48
clarkbjeblair: yup18:48
jeblairok, thx18:48
*** esker has joined #openstack-infra18:48
clarkbjeblair: when_it_happened.txt is the file18:48
*** dims_ has joined #openstack-infra18:49
*** hashar has joined #openstack-infra18:49
*** ZZelle_ has joined #openstack-infra18:50
clarkbjeblair: on saturday I restarted zuul manually to get past the issue18:50
clarkbdecided to do that after I saw the size of the log files18:50
clarkband it was halting stuff18:50
clarkbtoday I caught up and properly debugged it18:50
*** mpaolino has joined #openstack-infra18:51
*** melwitt has quit IRC18:51
*** melwitt has joined #openstack-infra18:51
*** e0ne has quit IRC18:52
jeblairclarkb: when_it_happened.txt sounds like a fanfic ebook from usenet.  i like it.  :)18:52
*** esker has quit IRC18:52
*** ramashri has quit IRC18:53
* fungi concatenates and uudecodes it for later reading18:53
*** james_li has joined #openstack-infra18:54
fungi(must have been compressed as a .Z file before posting... go figure)18:54
*** rainya has quit IRC18:55
*** e0ne has joined #openstack-infra18:56
*** achanda has quit IRC18:57
*** otherwiseguy has joined #openstack-infra18:59
*** mrmartin has quit IRC18:59
anteayajeblair: two items on tomorrow's agenda from the third-party meeting19:00
krtayloranteaya, whew, we always fill an hour, even with a sparse agenda  :)19:01
anteayakrtaylor: we do, lots of questions19:01
anteayayou run a good meeting, kurt, well done19:01
krtayloranteaya, thanks!19:01
beeceekrtaylor: to continue the discussion, which link were you referring to?19:01
anteayajeblair: tomorrow's infra-meeting agenda that is19:01
*** thedodd has quit IRC19:02
*** thedodd has joined #openstack-infra19:03
*** melwitt has quit IRC19:03
*** melwitt has joined #openstack-infra19:03
*** alexpilotti has quit IRC19:04
*** rainya has joined #openstack-infra19:04
*** ekarlso has quit IRC19:04
anteayabeecee: so are you familiar with this page? http://ci.openstack.org/third_party.html#requirements19:04
krtaylorbeecee, which driver will your CI system be testing?19:04
armaxclarkb: thanks for the reviews, I adjusted the patches to address your comments19:05
*** mpaolino has quit IRC19:05
clarkbthanks19:05
armaxclarkb: thank you19:05
beeceeanteaya: only in the last 2 weeks19:05
krtaylorbeecee, for instance, nova requirements are here: https://wiki.openstack.org/wiki/HypervisorSupportMatrix19:06
beeceekrtaylor: cinder/zadara .  we first heard about it from DuncanT-'s mail out, and my boss handed off the task to me, so I'm coming at this fresh19:06
jeblairclarkb: i'm a little worried that with your fix we could end up in an infinite reporting loop19:06
clarkbjeblair: hrm maybe. like if gerrit goes sideways19:07
krtaylorbeecee, understood, there are several -cinder groups starting to test drivers19:07
jeblairclarkb: there are two actions handled in the reporting: leaving a vote and verifying that the change merged (if applicable)19:07
clarkbor that change makes itself unhappy19:07
anteayabeecee: you hadn't heard about the need to test your driver prior to DuncanT's email?19:07
*** SumitNaiksatam has quit IRC19:07
anteayabeecee: just trying to understand your perspective here19:07
krtaylorI was about to ask the same19:07
anteayakrtaylor: also if you can review this series: https://review.openstack.org/#/q/status:open+project:openstack-infra/gerrit+branch:openstack/2.8.4,n,z19:08
clarkbjeblair I can look at it more in the afternoon19:08
anteayakrtaylor: this is to get me gerrit permissions to create and edit third party ci accounts19:08
clarkbit did bend my mind a bit this morning19:08
jeblairclarkb: in fact, i think your change would report results twice for the incident that triggered this19:08
jeblairclarkb: because it was the verification that failed, not the actual reporting19:09
clarkbright so there is a check to see if we reported19:09
beeceeanteaya: there's a difference between myself hearing about it, and the company hearing about it.  I hadn't, and from asking around internally, the people previously responsible for openstack integration hadn't either19:09
clarkbso it would report and succeed then verification failed19:09
anteayazaro: I think we have some rebase syntax in this file: https://review.openstack.org/#/c/109796/1/Documentation/cmd-set-account.txt19:09
anteayaline 26 <<<<<<< HEAD19:09
clarkbso next time through it skips reporting and verification succeeds19:09
jeblairclarkb: ah true, so it would not report twice19:10
anteayaand line 41 >>>>>>> aa49e27... Allow to delegate modify account commands to non administrators19:10
clarkbjeblair the important bit is the first part of reportItem()19:10
anteayabeecee: okay, so that is a gap we need to address19:10
krtayloranteaya, will do19:10
*** melwitt has quit IRC19:10
anteayabeecee: because it is hard to address something if you don't know about it19:10
jeblairclarkb: okay, so we may actually be sufficiently protected19:10
*** melwitt has joined #openstack-infra19:11
beeceeanteaya: in previous companies, I did CI stuff, so I'm personally in favour of having everything continually tested, but I'm chasing the formal rationale from openstack as to why actual $$ should be spent ;)19:11
anteayabeecee: so let's continue, so here you are, you found the third party meeting, well done and you know about the ci.openstack.org third party requirements19:11
anteayabeecee: got it19:11
anteayabeecee: let's start from your prior experience then19:11
anteayathat sounds like a happy place19:12
anteayaand then expand19:12
jeblairclarkb: okay, so i think we're only as suceptible to reporting loops as we are now (and that would be only if gerrit records a vote but still returns error).  i think that's okay.19:13
clarkbcool19:13
jeblairclarkb: if we wanted to be extra paranoid, we could eject the change from the queue with a try/finally around reportItem19:13
jeblairclarkb: but i don't think that's necessary19:13
jeblairat least, not yet :)19:14
*** pelix has quit IRC19:14
beeceeanteaya: the formal requirements that I've found are technical in nature.  the assumption is that you've woken up one day and said 'hey, I've got a 3rd party driver that I want to maintain in openstack, how do I go about testing it on every commit'19:15
*** htrmeira has joined #openstack-infra19:16
beeceeanteaya: I haven't found the political requirements, only hints that there are some and there are deadlines for test-it-or-lose-it19:16
*** ekarlso has joined #openstack-infra19:16
beeceeanteaya: and this may well be that I haven't been previously involved in openstack development19:17
*** melwitt has quit IRC19:17
*** melwitt has joined #openstack-infra19:17
anteayabeecee: political requirements19:17
anteayanot sure what you mean there19:17
htrmeiraHi guys, I am having some troubles creating a guest image for trove, does anyone know where can I find a tutorial?19:18
mordredanteaya: the ramifications of not chosing to test your driver on every commit19:18
mordredanteaya: the 'why' as it were19:18
anteayahtrmeira: have you spent time in the trove channel yet?19:18
anteayahtrmeira: #openstack-trove19:19
anteayamordred: ah19:19
*** melwitt has quit IRC19:19
htrmeiraNot yet. Thanks anteya. =)19:19
beeceemordred/anteaya: and the per-project cut-off dates for not doing so19:19
*** melwitt has joined #openstack-infra19:19
anteayabeecee: well you have it already, if you choose to not test your driver, it will eventually be deprecated from openstack19:19
beeceeanteaya: yes, I know that.  but where is that written?19:19
anteayabeecee: what project? cinder right?19:19
anteayawell it is just in the logs19:20
beeceeanteaya: cinder specifically, openstack in general19:20
anteayabeecee: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-07-28.log19:20
anteayaright19:20
anteayawell we haven't put it in front of the technical committee19:20
anteayasince right now we are working hard to get things shored up19:21
*** emagana has joined #openstack-infra19:21
anteayaand deadlines are on a per project basis19:21
anteayabut openstack's axiom is if it isn't tested, it's broken19:21
anteayaand the third party code didn't come in through the front door19:21
anteayais sort of eased in through the side, one project at a time19:22
openstackgerritKim Jensen proposed a change to openstack-infra/infra-manual: Added some initial content in Peer Review before ReviewChecklist link  https://review.openstack.org/10758819:22
openstackgerritAlexandre Viau proposed a change to openstack-infra/config: Added Jenkins jobs for Surveil  https://review.openstack.org/11009419:22
anteayauntil it threatened the stability of one notable project19:22
anteayaso openstack as a whole is trying to do this dance19:22
anteayaand right now everyone is doing it differently19:22
anteayaand we are on a spectrum, which we are trying to formalize19:22
anteayabut we aren't there yet19:22
anteayaand when we try to formalize too quickly we risk upsetting some folks, who want to play but didn't know what was expected of them19:23
anteayasuch as yourself19:23
anteayaso there is a balance19:23
anteayabut the quality was in question19:23
anteayaand needed to be addressed19:23
anteayaso actually a lot of this current situation comes from last release and the initiative of neutron19:24
anteayaand my stick weilding in that program19:24
krtayloryes, and there is a lot of "understood" info that is in the process of being written down, its getting better19:24
anteayaso we have started to get some process in place19:24
anteayanot fast enough for some, too fast for others19:24
anteayaso trying to keep the group together is a lot of running from edge to edge19:25
anteayaexhausting19:25
anteayaand if you wait for formalized policy you will honestly be missing the boat19:25
anteayajoin us and make the policy19:25
anteayadon't wait to read it19:25
*** melwitt has quit IRC19:26
beeceeanteaya: join the openstack side, we have cookies (and proto policy) ?19:26
anteayasort of yea19:26
anteayaif that sells you on it, we can put it on a t-shirt19:26
anteayaif not, I have more19:26
anteayabeen doing this dance almost a year now19:26
beeceeanteaya: sorry if I'm making you repeat this, but that's the core of my comments, finding out this stuff as a newbie is very hard19:27
anteayabut the heart of it is, if it isn't tested, it's broken19:27
*** yamamoto has joined #openstack-infra19:27
krtaylorhm, prob closer to "we have beer"19:27
anteayaand all else comes from that19:27
*** SumitNaiksatam has joined #openstack-infra19:27
anteaya'cept I don't drink, so beer doesn't sell me19:27
krtaylorgood for you19:28
anteayabeecee: I understand that19:28
anteayaI have no problem repeating it19:28
fungiclarkb: mordred: jeblair: if you find yourselves periodically getting unsubscribed from lists.o.o mls like i am, i've tracked down a significant cause... some worms are spoofing mailer-daemon and postmaster for that fqdn in list posts, mailman is rejecting the posts and bouncing them to *us* and them my mta (but also at least gmail) is rejecting them as containing malware, and mailman incorrectly19:28
fungiidentifies these as us bouncing ml post deliveries and increments our bounce scores19:28
anteayainteresting to have a chance to get a larger perspective once in a while19:28
anteayabeecee: and thanks for taking the time to ask19:28
anteayabeecee: are we getting any closer to getting you what you need?19:28
mordredfungi: uhm. wow19:28
fungiclarkb: mordred: jeblair: i'm not sure yet what the best solution to that is19:29
mordredfungi: nntp19:29
fungimordred: oh, right. problem solved!19:29
krtaylorbeecee, is anyone working on a cinder equivalent of the nova supported hypervisor matrix?19:30
krtayloralthough that is prob a -cinder question19:30
anteayakrtaylor: yeah that is probably a cinder question19:30
anteayakrtaylor: what matrix are you referencing? is it a wikipage?19:30
beeceekrtaylor: no idea.19:31
fungiclarkb: mordred: jeblair: i changed my exim config to start freezing those instead of rejecting them, and so far they have all been zip mime attachments identified by clamav as worm.mydoom-2719:31
*** yamamoto has quit IRC19:31
krtayloranteaya, https://wiki.openstack.org/wiki/HypervisorSupportMatrix19:32
griffkrtaylor: no, not really.  Cinder has a different view on that.  Closest we have is: https://wiki.openstack.org/wiki/CinderSupportMatrix19:32
openstackgerritA change was merged to openstack/requirements: Update oslo requirements to use juno dev versions  https://review.openstack.org/10933819:32
beeceekrtaylor: there's also a 3rd party matrix for cinder19:33
krtaylorgriff, very nice, I didn't know about that one19:33
beeceekrtaylor: https://wiki.openstack.org/wiki/Cinder/certified-drivers19:33
*** ildikov has quit IRC19:33
*** ildikov has joined #openstack-infra19:33
*** david-lyle has quit IRC19:34
*** alexpilotti has joined #openstack-infra19:34
*** terryw has joined #openstack-infra19:35
anteayagriff: yeah, when are we going to get the word certified removed from that wikipage?19:35
anteayagriff: do you have a timeline?19:35
anteayaI have asked cinder several times19:35
griffanteaya: as I said before I don't care... but if it makes people happy I'll update the wiki page right now19:36
anteayagreat thanks19:36
anteayait would make me happy19:36
griffanteaya: *who* is Cinder?  Never met him/her :)19:36
*** flaviof_zzz is now known as flaviof19:36
anteayawell the cinder team/program19:36
griffanteaya: I know.. I was joking19:36
fungicindy maccinder19:36
anteayagriff: kk19:36
grifffungi: :)19:37
anteayathe moments to successfully troll me are few and far between19:37
*** otherwiseguy has quit IRC19:37
*** cdub has quit IRC19:37
griffanteaya: what did you and the others decide would be an acceptable name?19:37
anteayaanything that means tested that isn't a version of certified19:38
anteayaI'll get teh link from the tc logs19:38
*** cdub has joined #openstack-infra19:38
*** _nadya_ has joined #openstack-infra19:38
griffanteaya: it's fine, I'll just change it to tested19:39
griffanteaya: IIRC you didn't like verified either19:39
*** ociuhandu has quit IRC19:39
beeceegriff: is the 'certified' from the so-called 'certification' test that driver_certs/cinder_driver_cert.sh runs?19:39
*** david-lyle has joined #openstack-infra19:39
griffbeecee: yes19:39
griffbeecee: and it was in fact a certification.  But please don't get that debate started again.  It's far too painful19:39
*** mrmartin has joined #openstack-infra19:40
anteayagriff: item 2.b: http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-06-17-20.03.html19:40
anteayabeecee: you can read the thread here: http://lists.openstack.org/pipermail/openstack-dev/2014-June/036933.html19:40
beeceegriff: that puts a crimp in my plans to print out our 'cert' test and frame it ;)19:41
anteayabeecee: and the meeting log here: http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-06-17-20.03.log.html19:41
*** zns has quit IRC19:42
anteayagriff: I don't recall expressing an opinion on verified, but tested was mentioned in the tc logs, so that is a direction I can support19:42
griffanteaya: updated, please check and give your approval if you have a moment19:42
anteayaI might have expressed an opinion on verified, I just don't recall doing so19:42
anteayagriff: what is the link?19:42
griffuse the old one, it should redirect19:43
anteayakk19:43
griffI'd like to test that as well19:43
* anteaya backscrolls19:43
griffJust ripping out wiki's isn't helpful, as some use bookmarks :)19:43
beeceegriff: redirect works19:43
griffbeecee: thanks!19:43
anteayano redirect for me19:43
* anteaya flushes firefox cache19:43
*** achanda has joined #openstack-infra19:44
anteayaurl still is https://wiki.openstack.org/wiki/Cinder/certified-drivers19:44
*** zns has joined #openstack-infra19:45
griffanteaya: that's because it's a redirect19:45
beeceegriff: and the wiki sent me two email notifications about it, guess I checked some notify-me box when I updated it19:45
griffanteaya: do you or do you not see the updated title and the removal of the word certified?19:45
griffbeecee: :)19:45
anteayaokay that works https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers19:45
anteayayes that works19:46
fungianteaya: note that its title is now "Cinder/tested-3rdParty-drivers" and below that it says "Redirected from Cinder/certified-drivers"19:46
griffanteaya: cool.  I'll remove the original link after a bit and replace it with a page that just says "not here anymore go here instead"19:46
fungiso yes, that latter url is now the canonical one19:46
anteayaas a small side note we are working toward using third party or third-party rather than 3rd party, to make searching and filtering easier19:46
openstackgerritA change was merged to openstack/requirements: Don't use SQLAlchemy 9 version < 0.9.7  https://review.openstack.org/10923819:46
griffanteaya: of course you are :)19:47
anteayabut that is more krtaylor bug, so I'll let him have that one19:47
anteayagriff: thanks for the update19:47
anteayait looks good19:47
*** alexpilotti has quit IRC19:48
*** andreaf has joined #openstack-infra19:48
*** _nadya_ has quit IRC19:49
anteayabeecee: so this is what the process looks like19:49
anteayabeecee: welcome19:49
anteayabeecee: let me know when you have enough to carry something to management19:50
*** jp_at_hp has quit IRC19:50
*** ihrachyshka has quit IRC19:51
*** lukego has quit IRC19:51
*** e0ne has quit IRC19:52
beeceeanteaya: still reading the certifed mail thread19:52
*** craigbr has quit IRC19:53
*** lukego has joined #openstack-infra19:53
*** achanda has quit IRC19:53
openstackgerritA change was merged to openstack-infra/zuul: Simplify change reporting to handle errors better  https://review.openstack.org/11006619:54
*** ihrachyshka has joined #openstack-infra19:54
fungiclarkb: ^ though i assume we don't need to rush to restart19:54
*** melwitt has joined #openstack-infra19:54
rcarrillocruzzaro: hey mate, are you around?19:56
*** melwitt has quit IRC19:56
anteayabeecee: make yourself some popcorn and take some water with you, it is a chewy chat19:56
*** melwitt has joined #openstack-infra19:56
*** e0ne_ has joined #openstack-infra19:56
anteayaokay I have to start getting packed19:57
anteayaso much for actually making slides today19:57
*** MarkAtwood has quit IRC19:57
anteayahope brisbane doesn't mind me just nattering on about something to do with logs19:57
anteayaI'm going with less clothes this time and laundry soap19:57
anteayalet's find out if this is a good direction19:58
*** e0ne_ has quit IRC19:58
*** melwitt has quit IRC19:59
*** melwitt1 has joined #openstack-infra19:59
*** cdub has quit IRC20:00
*** cdub has joined #openstack-infra20:00
*** andreykurilin has joined #openstack-infra20:01
*** marcoemorais has quit IRC20:01
*** marcoemorais has joined #openstack-infra20:01
*** andreaf_ has joined #openstack-infra20:03
*** achanda has joined #openstack-infra20:04
*** thedodd has quit IRC20:06
*** melwitt1 has quit IRC20:06
*** andreaf has quit IRC20:06
*** melwitt has joined #openstack-infra20:06
clarkbfungi probably not20:09
*** thedodd has joined #openstack-infra20:10
*** melwitt1 has joined #openstack-infra20:11
*** melwitt has quit IRC20:11
anteayabeecee: also something else to keep in mind is that customers/consumers of openstack compliant third party products are currently invited into the third party space20:13
openstackgerritA change was merged to openstack-infra/zuul: Continue reporting on URL format KeyError  https://review.openstack.org/10496320:13
anteayaand I am teaching them how to read the logs and assess the stability of code patches in gerrit as well as getting involved in reviews20:13
anteayabeecee: http://lists.openstack.org/pipermail/openstack-dev/2014-July/040648.html20:14
*** esker has joined #openstack-infra20:14
*** doddstack has joined #openstack-infra20:16
*** thedodd has quit IRC20:16
*** andreaf has joined #openstack-infra20:18
*** melwitt1 has quit IRC20:19
*** melwitt has joined #openstack-infra20:19
asselinclarkb, how does os-infra install rabbit mq needed by nodepool?20:19
clarkbrabbit mq?20:20
clarkbThere shouldnt be a rabbit20:20
*** juzuluag has quit IRC20:20
clarkbjust zeromq and gearman20:20
asselinclarkb, sorry...ok...zeromq...will search again20:20
*** andreaf_ has quit IRC20:20
anteayaasselin: let's try to not use os as a short form for openstack20:21
anteayaos means operating system20:21
*** e0ne has joined #openstack-infra20:21
asselinanteaya, ok good to know20:21
anteayaif you want to use o.o for the domain openstack.org that is something we are used to20:21
anteayaasselin: thanks20:21
asselinanteaya, thanks for the alternative20:21
anteayanp20:21
anteayafor some reason we are starting to see it and os always means operating system to me20:22
*** melwitt has quit IRC20:23
*** melwitt1 has joined #openstack-infra20:23
asselinyeah, there's a reason why they have dictionaries for acronyms20:24
fungiasselin: i believe it just gets pyzmq from pypi. i've seen cryptic issues arise from using it backed by a system install of libzmq1 instead of its internal implementation (which tends to happen if you install debian's python-zmq package for example)20:25
asselinfungi, ok thanks...looking for the o.o puppet script that installs it...hoping to just copy that20:25
*** melwitt1 has quit IRC20:26
fungiasselin: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/nodepool/manifests/init.pp#n8620:27
*** yamamoto has joined #openstack-infra20:27
fungiasselin: we have a puppet vcsrepo object which clones/updates the nodepool git repo and then a subscribed exec which calls pip install on it20:27
asselinfungi, that's what I used....20:28
fungiasselin: pip in turn pulls down any missing dependencies listed in http://git.openstack.org/cgit/openstack-infra/nodepool/tree/requirements.txt20:28
asselinfungi,  to install nodepool....how can I check if it's running?20:28
fungiasselin: service nodepool status20:29
*** pblaho has quit IRC20:29
asselinfungi,  * nodepool is running20:29
fungithere you go... off to the races20:29
openstackgerritRyan Petrello proposed a change to openstack-infra/config: Run pecan py33-scaffolds jobs on py3k-precise nodes.  https://review.openstack.org/10879820:29
asselinfungi,  ok I see..I though it was a separate process....will look into nodepool. thanks20:30
*** unicell has quit IRC20:30
*** melwitt has joined #openstack-infra20:30
fungiasselin: nodepoold will appear in your process list when it's running, and probably will correspond to a pid in /var/run/nodepool.pid20:30
fungisorry, /var/run/nodepool/nodepool.pid20:31
asselinfungi, I'm running nodepoold -d 'manually' as nodepool user....20:31
asselinI can try it as a service and see if that helps20:31
fungiasselin: oh, okay. i think that should work too if you want to run it in the foreground to debug20:31
*** yamamoto has quit IRC20:31
*** e0ne has quit IRC20:32
fungiasselin: running 'sudo nodepool image-list' and 'sudo nodepool list' will hopefully yield useful information if it's doing something, as will looking at its logs in /var/log/nodepool and the mysql db it's using to track status detauks20:33
fungidetails20:33
*** e0ne has joined #openstack-infra20:33
*** fifieldt has quit IRC20:34
*** cdub has quit IRC20:34
asselinfungi, I think I found the issue. I need to configure zmq-publishers correctly.20:34
asselinto be the same port as jenkins I assume20:35
*** shashankhegde has quit IRC20:35
*** david-lyle has quit IRC20:36
*** yfried_ is now known as yfried20:36
Ajaeger1SergeyLukjanov: is there any special reason that you +2ed but not approved the specs.o.o publishing patch https://review.openstack.org/#/c/106100 ?20:36
fungiasselin: yes, probably so. ours is a list of master urls like tcp://jenkins03.openstack.org:888820:37
*** Sukhdev has quit IRC20:38
asselinfungi, how do you know which port? is it the same as the jenkins http port?20:38
asselinfungi, or do you need a separate jenkins plugin for zmq?20:38
*** dimtruck is now known as zz_dimtruck20:39
fungiasselin: there is a jenkins zeromq publisher plugin20:39
*** julim has quit IRC20:39
*** melwitt has quit IRC20:39
*** cdub has joined #openstack-infra20:40
asselinfungi, ok thanks found it: Event Publisher (via ZMQ PUB SUB) 0.0.320:40
fungiasselin: zmq-event-publisher yep20:40
*** zz_dimtruck is now known as dimtruck20:40
*** david-lyle has joined #openstack-infra20:44
asselinfungi, thanks found it here: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/jenkins.pp#n8120:46
*** fifieldt has joined #openstack-infra20:46
asselinfungi, jeblair any idea when this will be released? https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/jenkins.pp#n8420:47
fungiasselin: i'm not certain what the latest thinking was there... i believe it's a fork of a deprecated/defunct plugin to which console log collection was added, but no idea if the idea is to recombine it or take it over or what20:52
*** yamahata_ has joined #openstack-infra20:53
fungii find old discussions about abandoning it in favor of the publish-over-ssh plugin, but its current state still seems to lack console log collection based on my (albeit cursory) review20:55
asselinfungi, ok I see. Do you know if anyone is working on finding a solution to this?20:56
fungiasselin: not to my knowledge. i think most current effort in openstack-infra is tending toward working on replacing jenkins rather than further improvements to jenkins plugins we're running20:57
asselinfungi, ok yes I remember that discussion. ok I'll leave it as-is then.20:58
*** cdub has quit IRC20:58
*** e0ne has quit IRC20:59
*** markmcclain1 has quit IRC21:00
*** markmcclain has joined #openstack-infra21:00
*** marcoemorais has quit IRC21:02
*** beecee has quit IRC21:04
*** beecee has joined #openstack-infra21:05
*** cdub has joined #openstack-infra21:05
*** mestery has quit IRC21:06
*** mestery has joined #openstack-infra21:06
*** Sukhdev has joined #openstack-infra21:08
*** msabramo has quit IRC21:08
openstackgerritA change was merged to openstack-infra/infra-specs: Added specification for storyboard subscription  https://review.openstack.org/9530721:10
*** zns has quit IRC21:12
*** yamamoto has joined #openstack-infra21:12
*** mfainberg_phone has joined #openstack-infra21:13
*** mfainberg_phone has quit IRC21:13
fungijenkins05 returned to the network on its own at 1930 utc. no indication in any of its logs that it thought anything was wrong (continued running cron jobs and whatnot during the outage) just like back on the 23rd21:13
*** zns has joined #openstack-infra21:13
Ajaeger1fungi: it finally had some storage on draft-docs ;)21:14
*** zns has quit IRC21:14
fungii'll gracefully restart the jenkins service on it and proceed with nodepool cleanup21:14
*** ArxCruz has quit IRC21:14
*** zns has joined #openstack-infra21:14
fungi(graceful jenkins shutdown, delete all jenkins05 nodepool nodes, start jenkins service again)21:14
*** htrmeira has quit IRC21:16
beeceefungi: I've sometimes seen machines become invisible to the network because their MAC address expired from their local switch because the machine wasn't sending any traffic to refresh the entry.21:17
*** james_li has quit IRC21:18
fungibeecee: not the case here... these servers are in constant communication with one another (cacti server polling snmp on them, puppet master logging in to apply manifests, jenkins communicating build status and jobs via zmq and gearman to zuul/nodepool, slave agent keepalives to all connected slaves, et cetera)21:19
*** prad has quit IRC21:19
*** mrmartin has quit IRC21:19
*** marcoemorais has joined #openstack-infra21:19
*** shashankhegde has joined #openstack-infra21:20
*** e0ne has joined #openstack-infra21:22
*** e0ne has quit IRC21:23
*** lukego has quit IRC21:25
*** juzuluag has joined #openstack-infra21:25
*** Ajaeger1 has quit IRC21:26
*** cdub has quit IRC21:28
*** eharney has quit IRC21:28
jeblairasselin: i think we broke backwards compatability in the scp plugin21:30
jeblairasselin: so i don't think we should release it until it's fixed21:30
jeblairasselin: i don't think anyone is working on fixing it21:31
*** melwitt has joined #openstack-infra21:31
jeblairi'm also a little uncomfortable releasing it because the licensing situation is not clear21:31
jeblairnot sure i want to put my name on that :)21:31
asselinjeblair, ok makes sense21:31
*** garyh has quit IRC21:32
*** garyh has joined #openstack-infra21:33
*** mwagner_lap has quit IRC21:35
*** dustins has quit IRC21:35
*** msabramo has joined #openstack-infra21:35
*** signed8bit has quit IRC21:36
*** flaper87 is now known as flaper87|afk21:36
*** melwitt has quit IRC21:36
*** melwitt has joined #openstack-infra21:37
*** esker has quit IRC21:38
*** melwitt1 has joined #openstack-infra21:39
*** ihrachyshka has joined #openstack-infra21:39
*** melwitt has quit IRC21:39
*** melwitt1 has quit IRC21:40
*** melwitt has joined #openstack-infra21:40
jogographite logs look a bit funny today http://jogo.github.io/gate/21:41
*** jgrimm has quit IRC21:41
*** pdmars has quit IRC21:41
*** weshay has quit IRC21:42
*** pdmars has joined #openstack-infra21:42
*** eharney has joined #openstack-infra21:43
anteayajogo: static.o.o filled up for a bit off the top of the day21:44
*** andreaf has quit IRC21:44
anteayajogo: might that explain the 100% failure rate?21:44
jogoanteaya: hmm yeah that may be it, thanks21:45
anteayanp21:45
anteayajogo: how is nova meetup?21:45
anteayaand when do you land in brisbane :D21:46
anteayaha ha ha21:46
*** dkranz has quit IRC21:46
jogoway too soon21:46
anteayaha ha ha21:46
jogonova meetup is good21:46
anteayacool21:46
*** daya_k has quit IRC21:46
jogooh I have some slides for you21:46
jogoi'll email them21:46
*** james_li has joined #openstack-infra21:46
anteayado you21:46
anteayaI'll take slides21:46
anteayathanks21:46
*** dhellmann_ has joined #openstack-infra21:47
anteayajogo: oh and markmcclain is trying to figure out migrations so they work with neutron and nova21:47
anteayajogo: you'll help him with that, will you?21:47
*** pdmars has quit IRC21:47
*** dhellmann has left #openstack-infra21:47
anteayajogo: which email inbox should I be staring at?21:48
*** msabramo has quit IRC21:48
anteayaah I see them, thanks21:49
*** msabramo has joined #openstack-infra21:50
*** flaviof is now known as flaviof_zzz21:50
anteayajogo: well from slide 5 I can focus on the logs for those tests21:50
anteayathat sets you up well21:50
jogoanteaya: yeah21:50
anteayaparticularly tempest21:50
anteayasince unit tests logs can be a bit all over the place21:51
jogoyeah there can be some good tie ins21:51
anteayabut I can give them a hat tip21:51
*** flaviof_zzz is now known as flaviof21:51
anteayayes, having your slides helps thanks21:51
jogoanteaya: your talk is right before mine, right21:51
jogoanteaya: great21:51
anteayawell sdague's slides, arent' they21:51
jogoanteaya: sorry for taking so long to send 'em21:51
jogoanteaya: shhh21:51
anteayajogo: yes, I am your warm-up act21:51
*** msabramo has left #openstack-infra21:51
anteayajogo: :D21:51
jogoand yes thank you sdague21:51
anteayajogo: np, I don't even have mine started21:51
anteayaso I'll just focus on setting up your slides21:52
anteayamakes it consistent for the audience21:52
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova-network floating IPs bug 1349590  https://review.openstack.org/11014221:52
uvirtbotLaunchpad bug 1349590 in nova "nova-network appears to lock up while associating floating ip's" [High,Confirmed] https://launchpad.net/bugs/134959021:52
anteayanice graphics21:53
anteayawhat app was used, inkscape?21:53
jogoanteaya: which graphics?21:53
jogoall were open office I think21:53
anteayaslide 1621:53
anteayareally, open office does that?21:53
anteayaI am impressed21:53
anteayayay I got a mention on slide 22!21:54
anteayathank you21:54
jogohehe thank sdague too21:54
jogo(thank him for most if it actually)21:54
anteayaha ha ha21:55
anteayaI think we each have half an hour or something21:56
anteayawe can arm wrestle for time once you are jet-lagged in brisbane21:57
*** juzuluag has quit IRC21:57
anteayaI'll win21:57
*** harlowja is now known as harlowja_away21:57
*** dhellmann_ is now known as dhellmann21:57
*** ihrachyshka has quit IRC21:57
*** dhellmann_ has joined #openstack-infra21:58
*** dhellmann_ has quit IRC21:58
*** skath has left #openstack-infra21:59
*** melwitt has quit IRC21:59
*** melwitt1 has joined #openstack-infra21:59
*** skath has joined #openstack-infra22:02
*** stevemar has quit IRC22:02
*** harlowja_away is now known as harlowja22:03
*** zns has quit IRC22:04
*** marcoemorais has quit IRC22:04
*** marcoemorais1 has joined #openstack-infra22:04
*** marcoemorais1 has quit IRC22:05
*** marcoemorais has joined #openstack-infra22:05
*** melwitt1 has quit IRC22:05
*** zns has joined #openstack-infra22:05
*** melwitt has joined #openstack-infra22:05
*** amcrn has quit IRC22:06
*** markmcclain has quit IRC22:06
*** melwitt has quit IRC22:06
*** melwitt has joined #openstack-infra22:06
openstackgerritDavid Lenwell proposed a change to openstack-infra/infra-manual: Changed width of the image at the top of the page  https://review.openstack.org/10886722:09
*** melwitt1 has joined #openstack-infra22:09
*** harlowja has quit IRC22:10
*** melwitt has quit IRC22:10
*** melwitt1 has quit IRC22:10
*** melwitt has joined #openstack-infra22:10
*** flaviof is now known as flaviof_zzz22:10
*** harlowja has joined #openstack-infra22:11
*** marcoemorais has quit IRC22:12
*** marcoemorais has joined #openstack-infra22:13
*** melwitt has quit IRC22:14
*** melwitt has joined #openstack-infra22:14
*** armax has left #openstack-infra22:15
*** armax has joined #openstack-infra22:15
*** armax has quit IRC22:16
mattoliverauMorning all22:16
anteayamorning mattoliverau22:16
mattoliveraujogo, anteaya: you both ready for your flights down under? :)22:18
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: Install modules before running puppet apply test  https://review.openstack.org/11014522:18
jesusaurusclarkb: ^^22:18
*** melwitt has quit IRC22:19
*** melwitt has joined #openstack-infra22:19
anteayamattoliverau: I leave the house tomorrow at 9am22:19
anteayaworking on getting my things packed22:19
anteayano idea when jogo's flight leaves22:19
anteayahe is in oregon22:19
clarkbjesusaurus: that may just break if we are in a weird place. that is fine I guess and would confirm more weirdness22:20
anteayamattoliverau: the weather looks like it is sunny and in the 20's every day all week22:20
anteayais kim jensen in channel?22:21
openstackgerritMaru Newby proposed a change to openstack-infra/config: Fix neutron func job to install neutron and deps  https://review.openstack.org/11014722:21
mattoliverauAnteaya, cool, it should be nice weather, Brisbane in winter is lovely and warm :)22:21
anteayawould be nice if she could respond to my comment on her -1: https://review.openstack.org/#/c/104951/22:21
anteayamattoliverau: nice22:21
*** pcm_ has quit IRC22:21
anteayamattoliverau: to be honest, I still can't even think about it, I'm functioning on autopilot22:22
jesusaurusclarkb: yeah, it would also uncover issues where we make a change to a module version, then an incompatible change gets merged after the module change but before nodepool updates its images22:22
mattoliverauanteaya: I think jogo traveling with Mikal as he is going from mid cycle to pycon, I think.22:23
jogomattoliverau: I am on mikals flight22:23
*** hashar has quit IRC22:23
jogomattoliverau: it makes lif easy22:23
mattoliverauanteaya: with all the traveling you've done recently I'm not surprised22:23
anteayamattoliverau: and more to come, 3 this trip22:24
anteayamy longest yet22:24
anteayajogo: nice, when do you land?22:24
mattoliveraujogo: nice, and you get an endless supply of mikal patented sarcasm the whole way, so worth every dollar ;)22:25
*** melwitt has quit IRC22:25
*** masayukig has quit IRC22:26
anteayaha ha ha22:26
anteayai want to hear mikal giggle for 20 minutes again22:26
anteayamade the summit for me22:27
*** dangers_away is now known as dangers22:28
mattoliverauanteaya: lol, yeah, make sure you bring a USB pet rock :)22:28
jogoanteaya: the morning of my talk22:30
jogomattoliverau: amen22:30
*** camunoz has joined #openstack-infra22:30
*** masayukig has joined #openstack-infra22:33
anteayamattoliverau: ha ha ha22:33
anteayajogo: you will be one bleary eyed boy22:34
anteayahopped up on caffiene22:34
anteayanot sure I will be able to be in the same room as you22:34
anteayamattoliverau: ah just invoking the rock does it22:34
anteayaha ha ha22:34
beeceeI know I'm going to regret asking this, but 'usb pet rock' ?22:35
*** terryw has quit IRC22:35
beeceeWas that another bit of silliness from the last summit?22:36
anteayawell it went like this22:36
anteayaso on the saturday after summit me and the australians went to the atlanta aquarium22:36
*** camunoz has quit IRC22:36
anteayaand I loved it, and they had to keep fetching me22:36
anteayawe had a great time22:36
anteayawe went to the gift shop22:36
anteayaand pleia2 had been sick and couldn't make it to summit, so I bought her a stuffed otter by way of saying we miss you22:37
anteayaand so they were waiting for me in the gift shop22:37
anteayaso then I finished and we left22:37
*** dkliban has quit IRC22:37
anteayaturns out jhesketh had also made a purchase22:37
anteayaa rock22:37
anteayawith a usp cable stuck to it22:37
anteayain a box, with a booklet of instructions22:38
anteayaand mikal lost his mind22:38
anteayacouldn't stop giggling22:38
anteayafor like 20 minutes22:38
anteayabest time of the summit22:38
nibalizerwhat did lsusb show?22:38
anteayaevery time he was in danger of stopping he would ask jhesketh a question about the purchase22:38
anteayajhesketh: ^^ ?22:39
nibalizerwe need to know22:39
anteayalike how much he paid for it22:39
anteayajhesketh said it was on sale22:39
*** mfink has quit IRC22:39
anteayathat was good for about another 5 minutes22:39
anteayamikal also asked if there was a book of instructions22:39
anteayathere was22:39
anteayait was blank pages22:39
jogoanteaya: yeah, it will be interesting, that is for sure22:40
anteayathat was good for a few minutes as well22:40
beeceeanteaya: not the strangest usb thing I've seen.  skymall back from atlanta offered a 'usb paper towel holder' with an interchangable wine stopper as an ornamental feature on top22:40
anteayabeecee: ha ha ha22:40
anteayabeecee: like because when I have wet hands needing usb activity is my first though22:40
anteayat22:40
beeceebut.. a rock.22:42
openstackgerritA change was merged to openstack-infra/config: Run pecan py33-scaffolds jobs on py3k-precise nodes.  https://review.openstack.org/10879822:43
jheskethMorning22:43
*** emagana has quit IRC22:43
*** atiwari has quit IRC22:43
*** doddstack has quit IRC22:45
*** mfink has joined #openstack-infra22:45
anteayamorning jhesketh22:45
anteayajust telling beecee and nibalizer the story of the rock22:46
*** emagana_ has joined #openstack-infra22:46
anteayafrom the aquarium22:46
*** fifieldt_ has joined #openstack-infra22:46
*** armax has joined #openstack-infra22:46
jheskethHaha, it was wui22:47
jhesketh*well worth it :-)22:47
*** mwagner_lap has joined #openstack-infra22:47
*** adalbas has quit IRC22:49
mordredjhesketh: when you plug it in to usb - what does it do?22:49
*** datsun180b has quit IRC22:50
*** dmsimard is now known as dmsimard_away22:50
jheskethAs advertised in the blank instruction booklet... Nothing22:50
openstackgerritMaru Newby proposed a change to openstack-infra/config: Fix neutron func job to use gate-specific tox env  https://review.openstack.org/10403922:50
jheskethdmesg, lsusb etc all showed blank22:51
*** craigbr has joined #openstack-infra22:51
anteayastill has me giggling22:54
*** nelsnelson has quit IRC22:54
*** emagana_ has quit IRC22:57
anteayait just occured to me I don't have any hotels booked yet22:57
anteayaI should do that22:57
*** james_li has quit IRC22:59
*** ZZelle_ has quit IRC22:59
*** vhoward has joined #openstack-infra23:01
*** r1chardj0n3s_afk is now known as r1chardj0n3s23:01
*** camunoz has joined #openstack-infra23:02
*** emagana has joined #openstack-infra23:03
*** andreykurilin has quit IRC23:04
*** marcoemorais has quit IRC23:04
*** dimtruck is now known as zz_dimtruck23:05
*** marcoemorais has joined #openstack-infra23:05
*** marcoemorais has quit IRC23:06
*** marcoemorais has joined #openstack-infra23:07
*** Sukhdev has quit IRC23:07
*** emagana has quit IRC23:08
*** rfolco has quit IRC23:08
fungiokay, jenkins05 has been gracefully stopped, all associated nodes deleted from nodepool/jenkins and then started again. i'm going to make a pass to clean up alien nodepool orphans in our providers too while i'm at it23:09
anteayaany indication what happened to jenkins05?23:11
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Formalized Resource Criteria API  https://review.openstack.org/11016023:11
fungianteaya: nope. it just seems to have stopped receiving any tcp/udp traffic for nearly 30 hours and then magically reappeared. so far rackspace hasn't picked up the support ticket i opened earlier in the day23:12
*** SumitNaiksatam has quit IRC23:13
anteayawow23:14
anteayaseems it needed a bit of a vacation23:15
anteayais it happier now?23:15
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Formalized Resource Criteria API  https://review.openstack.org/11016023:15
fungianteaya: seems to be. has fresh nodepool nodes and is running jobs23:15
fungianteaya: though if it starts piling on ready nodes which are actually used, then i guess we're looking at a nodepool restart23:16
openstackgerritDavid Lenwell proposed a change to openstack-infra/config: removed py3 tests from refstack-client  https://review.openstack.org/10944323:17
anteayainteresting23:17
fungisince that'll be a sign nodepool isn't watching its zmq events23:17
anteayaare those 2 33 hour jobs at the top of check from when jenkins05 disappeared23:18
fungiyeah23:18
anteayafungi: hmmmm23:18
anteayakk23:18
fungiif they get new patchsets, or if zuul gets restarted, they'll clear up23:18
*** craigbr has quit IRC23:18
anteayalike stones rolling around in the dryer23:18
openstackgerritDavid Lenwell proposed a change to openstack-infra/infra-manual: Add an introduction paragraph to the developer guide  https://review.openstack.org/10702423:20
fungiyep, that is indeed the case23:22
clarkbfungi: so you need to restart nodepool?23:22
fungistopping jenkins05 again so it doesn't accumulate many more23:22
fungiclarkb: yeah :(23:22
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Formalized Resource Criteria API  https://review.openstack.org/11016023:23
*** fifieldt_ has quit IRC23:23
fungiin another minute or two i'll nodepool delete all the ones which aren't running jobs23:23
openstackgerritDavid Lenwell proposed a change to openstack-infra/infra-manual: added link to git-review readme  https://review.openstack.org/10885723:25
*** dangers is now known as dangers_away23:25
clarkbfungi: ok23:25
*** antonym has quit IRC23:26
*** shashankhegde has quit IRC23:27
*** praneshp has quit IRC23:30
*** david-lyle has quit IRC23:31
*** jamielennox|away is now known as jamielennox23:32
*** david-lyle has joined #openstack-infra23:32
openstackgerritA change was merged to openstack-infra/config: Fix neutron func job to install neutron and deps  https://review.openstack.org/11014723:33
*** zz_dimtruck is now known as dimtruck23:33
*** achanda has quit IRC23:35
*** bknudson has quit IRC23:36
*** craigbr has joined #openstack-infra23:36
*** david-lyle has quit IRC23:36
*** achanda has joined #openstack-infra23:37
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for grenad ssh protocol banner bug 1349617  https://review.openstack.org/11016423:39
uvirtbotLaunchpad bug 1349617 in grenade "test_volume_boot_pattern fails in grenade with "SSHException: Error reading SSH protocol banner[Errno 104] Connection reset by peer"" [Undecided,New] https://launchpad.net/bugs/134961723:39
*** cnesa6 has quit IRC23:40
*** antonym has joined #openstack-infra23:40
*** maxbit has joined #openstack-infra23:42
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Updated story list interface.  https://review.openstack.org/11016523:42
*** maxbit has quit IRC23:43
*** fifieldt_ has joined #openstack-infra23:45
*** fifieldt has quit IRC23:45
*** SumitNaiksatam has joined #openstack-infra23:48
*** craigbr has quit IRC23:50
*** otherwiseguy has joined #openstack-infra23:52
openstackgerritpatrick-crews proposed a change to openstack-infra/config: Adding tripleo-ci files to be indexed by logstash  https://review.openstack.org/10784123:57
*** banix has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!