| openstackgerrit | A change was merged to openstack-infra/config: Rename gertty ACL file to correct name https://review.openstack.org/91197 | 00:00 |
|---|---|---|
| *** morgabra has quit IRC | 00:01 | |
| harlowja | jeblair haha, u'll start love widget = widget.contents[0][0] | 00:02 |
| harlowja | :-P | 00:02 |
| harlowja | that part i don't like | 00:02 |
| harlowja | i had to do that weirdness to | 00:02 |
| *** marcoemorais has quit IRC | 00:05 | |
| sdague | notmyname: what defines your waiting to merge link? | 00:06 |
| dims | hmm, are we missing work in progress button? | 00:06 |
| sdague | dims: it's now workflow -1 | 00:06 |
| *** yamahata has joined #openstack-infra | 00:06 | |
| dims | ah. thx | 00:06 |
| derekh | fungi: found it, the rackspace fedora image uses a different grub config to the default fedora cloud image, running some tests now | 00:08 |
| fungi | derekh: fun! | 00:09 |
| fungi | derekh: i've noticed that rs, in general, does not use vanilla "cloud" images in their catalog | 00:09 |
| *** morgabra has joined #openstack-infra | 00:10 | |
| derekh | fungi: yup | 00:13 |
| *** atiwari has quit IRC | 00:14 | |
| sdague | notmyname: looking at waiting for merge, I feel like we probably need a set of queries which are mostly house keeping. Sets of odd states that we don't think things should in | 00:15 |
| *** sacharya has joined #openstack-infra | 00:16 | |
| *** sacharya has quit IRC | 00:16 | |
| sdague | jeblair: so what's your thinking around the fact that label:Code-Review>=0,self throwing an error if not logged in on that dashboard | 00:17 |
| *** SumitNaiksatam has quit IRC | 00:18 | |
| *** nati_ueno has joined #openstack-infra | 00:20 | |
| *** reed has quit IRC | 00:21 | |
| *** reed has joined #openstack-infra | 00:21 | |
| *** manishg has joined #openstack-infra | 00:22 | |
| *** nati_uen_ has quit IRC | 00:23 | |
| jeblair | sdague: if i click on the important changes dashboard while not logged in, it says my session is expired. i find it curious that the error is differnt. | 00:24 |
| jeblair | sdague: but i think i agree with your earlier sentiment -- there's probably nothing for it and we'll just have to accept it as is. | 00:25 |
| *** reed has quit IRC | 00:25 | |
| *** manishg_ has joined #openstack-infra | 00:25 | |
| *** manishg has quit IRC | 00:26 | |
| openstackgerrit | A change was merged to stackforge/gertty: Initial commit https://review.openstack.org/91191 | 00:27 |
| *** yidclare has quit IRC | 00:28 | |
| *** thomasem has joined #openstack-infra | 00:29 | |
| *** talluri has joined #openstack-infra | 00:31 | |
| *** praneshp has quit IRC | 00:31 | |
| harlowja | durn just added comments to that, lol | 00:32 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Use grub2-mkconfig if update-grub isn't present https://review.openstack.org/91188 | 00:32 |
| harlowja | anyway, nothing to bad, jeblair u urwid expert now, lol | 00:32 |
| jeblair | harlowja: oops, sorry, i'll go check them out, thanks | 00:33 |
| harlowja | np | 00:33 |
| harlowja | :) | 00:33 |
| jeblair | harlowja: and it's terrifying that you think i'm a urwid expert ;) | 00:33 |
| harlowja | hahaha | 00:33 |
| sdague | jeblair: I'm surprised you didn't write it in elisp as an emacs extension | 00:33 |
| sdague | :) | 00:33 |
| derekh | fungi: ^ that looks like it works on the rackspace fedora | 00:33 |
| harlowja | jeblair u seem to have found most of the urwid oddness :-P | 00:34 |
| fungi | derekh: oh, cool | 00:34 |
| harlowja | so that makes u expert, lol | 00:34 |
| harlowja | and u even got popups | 00:34 |
| jeblair | sdague: i thought about it... :) | 00:34 |
| sdague | jeblair: I would support that :) | 00:35 |
| sdague | or a server that exposes it as IMAP :) | 00:35 |
| fungi | derekh: oh, and yeah, i had to jump through similar hoops upgrading rackspace's debian wheezy images to jessie since they stub in an old grub. must be something about their xen domu boot setup | 00:35 |
| derekh | fungi: yup must be | 00:36 |
| openstackgerrit | Joshua Harlow proposed a change to stackforge/gertty: Read using a file handle instead of read() method https://review.openstack.org/91204 | 00:42 |
| harlowja | alright jeblair https://review.openstack.org/91204 | 00:42 |
| harlowja | :) | 00:43 |
| harlowja | ha | 00:43 |
| fungi | jeblair: if https://review.openstack.org/91188 seems sane to you, i'll babysit centos6 and precise image updates to make sure there are no unfortunate surprises | 00:43 |
| openstackgerrit | Joshua Harlow proposed a change to stackforge/gertty: Allow for password prompting https://review.openstack.org/91205 | 00:50 |
| *** tteggel has quit IRC | 00:52 | |
| *** UtahDave has quit IRC | 00:53 | |
| *** Adri2000 has quit IRC | 00:53 | |
| *** jhesketh has quit IRC | 00:53 | |
| openstackgerrit | Joshua Harlow proposed a change to stackforge/gertty: Add ordereddict requirement https://review.openstack.org/91206 | 00:53 |
| *** stevebaker has quit IRC | 00:53 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add tool to parse puppet reports https://review.openstack.org/91207 | 00:54 |
| mordred | jeblair, fungi ^^ | 00:54 |
| mordred | that will parse a puppet yaml report file and print some basic things | 00:54 |
| *** tteggel_ has joined #openstack-infra | 00:54 | |
| mordred | but more importantly, it will parse the file into some objects | 00:54 |
| openstackgerrit | Joshua Harlow proposed a change to stackforge/gertty: Add ordereddict requirement https://review.openstack.org/91206 | 00:54 |
| *** Adri2000 has joined #openstack-infra | 00:55 | |
| *** stevebaker has joined #openstack-infra | 01:00 | |
| *** nati_ueno has quit IRC | 01:01 | |
| openstackgerrit | David Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Doc: Use VersionInfo from pbr and remove openstack common modules https://review.openstack.org/90658 | 01:01 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix "delete-all" https://review.openstack.org/91208 | 01:02 |
| openstackgerrit | Mathew Odden proposed a change to openstack-infra/config: Create bash8 project in openstack-dev https://review.openstack.org/88747 | 01:04 |
| *** nati_ueno has joined #openstack-infra | 01:04 | |
| *** jhesketh has joined #openstack-infra | 01:08 | |
| *** dkehn_afk is now known as dkehn | 01:08 | |
| derekh | fungi: I gotta call it a night, I know of any other problems and will check in first thing to help deal with anything that might crop up | 01:11 |
| derekh | *I don't know of any .... | 01:13 |
| *** derekh has quit IRC | 01:13 | |
| tchaypo | dstufft: finally have a new commit with tests - let me know if you'd prefer the two commits squashed | 01:16 |
| jeblair | fungi: lgtm | 01:16 |
| dstufft | tchaypo: nah, we don't really care about squashing | 01:16 |
| tchaypo | dstufft: that was a quick response | 01:17 |
| jeblair | mordred, krotscheck: is there a way to get the task id in storyboard? | 01:17 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Update zuul merger init script https://review.openstack.org/91211 | 01:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Use grub2-mkconfig if update-grub isn't present https://review.openstack.org/91188 | 01:19 |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Fix displaying new files https://review.openstack.org/91212 | 01:19 |
| *** lakshmi has joined #openstack-infra | 01:20 | |
| dstufft | tchaypo: looks reasonable to me, did you ensure the tests fail w/o your change? | 01:20 |
| openstackgerrit | A change was merged to stackforge/gertty: Read using a file handle instead of read() method https://review.openstack.org/91204 | 01:20 |
| openstackgerrit | A change was merged to stackforge/gertty: Allow for password prompting https://review.openstack.org/91205 | 01:21 |
| tchaypo | dstufft: the only one that fails without my change is test_url_incorrect_case_file_index | 01:21 |
| mordred | jeblair: unsure. | 01:21 |
| dstufft | tchaypo: works for me | 01:22 |
| mordred | jeblair: how do I get help text in an xterm wthat 'helpfully' thinks F1 is for it? | 01:22 |
| openstackgerrit | A change was merged to stackforge/gertty: Add ordereddict requirement https://review.openstack.org/91206 | 01:22 |
| dstufft | tchaypo: I just wanted to make sure you verified that so that we know that there is a correct test for it so this stuff doesn't break in the future :] | 01:22 |
| jeblair | mordred: that's apalling. | 01:22 |
| mordred | jeblair: I just changed the setting in the terminal | 01:22 |
| mordred | nevermind | 01:22 |
| tchaypo | Yeah. Having a test that passes after the change because it's a broken test is quite pointless :) | 01:23 |
| openstackgerrit | A change was merged to stackforge/gertty: Fix displaying new files https://review.openstack.org/91212 | 01:23 |
| *** dkehn is now known as dkehn_zzz | 01:23 | |
| dstufft | tchaypo: ya, folks do it so I ask! | 01:24 |
| dstufft | I'll merge once tests pass | 01:24 |
| openstackgerrit | Joshua Harlow proposed a change to openstack/requirements: Add wrapt library https://review.openstack.org/87395 | 01:24 |
| jeblair | mordred: i think HEAD may be just barely usable for you now | 01:24 |
| dstufft | tchaypo: uh oh | 01:24 |
| dstufft | tchaypo: failing tests | 01:24 |
| jeblair | mordred: though heads up about this bug: https://storyboard.openstack.org/#!/story/74 | 01:25 |
| dstufft | tchaypo: looks like it fails on everything but 2.7 | 01:25 |
| tchaypo | well that's rude. | 01:25 |
| *** talluri has quit IRC | 01:25 | |
| dstufft | https://travis-ci.org/pypa/pip/builds/24066032 | 01:25 |
| tchaypo | oh. coincidentally, I only tested it on 2.7 | 01:25 |
| dstufft | oh .message is deprecated in 2.x and removed in 3.x | 01:25 |
| *** saschpe has quit IRC | 01:26 | |
| mordred | sqlalchemy.exc.OperationalError: (OperationalError) database is locked u'UPDATE project SET subscribed=? WHERE project."key" = ?' (1, 157) | 01:27 |
| tchaypo | actually according to https://travis-ci.org/pypa/pip/jobs/24066036 it's deprecated as of 2.6 | 01:27 |
| tchaypo | so I'm confused about why 2.7 works | 01:27 |
| dstufft | probably they screwed up the deprecation message on 2.7 | 01:28 |
| *** saschpe has joined #openstack-infra | 01:30 | |
| *** david-lyle has joined #openstack-infra | 01:32 | |
| *** Daviey has quit IRC | 01:33 | |
| tchaypo | on a side note - when I run "tox -epy27" I get a ton of failures. I've been ignoring everything that wasn't related to the code I was working on. Is there something I'm missing about running the tests locally? | 01:33 |
| *** mfer has joined #openstack-infra | 01:33 | |
| *** talluri has joined #openstack-infra | 01:34 | |
| *** melwitt has joined #openstack-infra | 01:35 | |
| *** lakshmi has quit IRC | 01:35 | |
| *** nosnos has joined #openstack-infra | 01:35 | |
| *** talluri has quit IRC | 01:37 | |
| *** talluri has joined #openstack-infra | 01:37 | |
| *** Ryan_Lane has quit IRC | 01:37 | |
| *** talluri has quit IRC | 01:41 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Install bootstrap for zuul module https://review.openstack.org/91214 | 01:41 |
| *** SumitNaiksatam has joined #openstack-infra | 01:42 | |
| *** krtaylor has joined #openstack-infra | 01:42 | |
| *** sandywalsh has quit IRC | 01:51 | |
| *** yaguang has joined #openstack-infra | 01:54 | |
| portante | hello folks, how do I go about adding another test run for the swift project? | 01:55 |
| portante | We recently added the ability to run our functional test suite entirely within a single process (thanks eventlet), so that now one can do: "tox -e pep8,py26,py27,func" | 01:56 |
| *** david-lyle has quit IRC | 01:57 | |
| portante | while we there is a devstack based functional test run as a check/gate job, I'd like to see this new in-process functional test run executed as well. | 01:57 |
| portante | notmyname: do you agree? | 01:57 |
| *** mfer has quit IRC | 01:59 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Install bootstrap for zuul module https://review.openstack.org/91214 | 02:01 |
| *** homeless has quit IRC | 02:05 | |
| *** zehicle_at_dell has quit IRC | 02:06 | |
| *** manishg_ has quit IRC | 02:07 | |
| *** zehicle_at_dell has joined #openstack-infra | 02:07 | |
| *** thomasem has quit IRC | 02:08 | |
| *** arnaud has quit IRC | 02:09 | |
| *** wenlock has joined #openstack-infra | 02:12 | |
| *** zhiyan_ is now known as zhiyan | 02:12 | |
| *** ptoohill has joined #openstack-infra | 02:24 | |
| ptoohill | Hello, following articles about error led me here. Im currently having issues submitting my first change with the following error from git-review: fatal: ICLA contributor agreement requires current contact information. I believe i may need to resign contract, but am unsure how to proceed. Any advice? | 02:25 |
| *** mriedem has quit IRC | 02:26 | |
| ptoohill | Also, to note, when attempting to update contact info in gerrit i get a server error "Cannot store contact information" | 02:26 |
| *** harlowja is now known as harlowja_away | 02:27 | |
| ptoohill | Should i just create a new account? or maybe im asking in the wrong place | 02:29 |
| *** crc32 has joined #openstack-infra | 02:29 | |
| *** melwitt has quit IRC | 02:32 | |
| *** jhesketh has quit IRC | 02:32 | |
| crc32 | Gerrit is complaining that a coworker hasen't set up their contact info and is rejecting their commits for review. This seems to be along the lines of https://ask.openstack.org/en/question/3413/icla-contributor-agreement-requires-current-contact-information-error-while-using-git-review | 02:32 |
| crc32 | is their a work around for this? | 02:32 |
| *** jhesketh has joined #openstack-infra | 02:32 | |
| crc32 | where are the admins to contact? | 02:32 |
| *** jhesketh has quit IRC | 02:33 | |
| mattoliverau | ptoohill: you are in the right place.. unfortuantly this week is the openstack off week (between releases) so many of the cores are not around. | 02:35 |
| ptoohill | gotcha | 02:35 |
| ptoohill | Any known work-arounds? | 02:35 |
| ptoohill | that you know of? | 02:35 |
| *** melwitt has joined #openstack-infra | 02:36 | |
| clarkb | first go to your gerrit settings and submit contact info as that seems to line up with your error | 02:36 |
| openstackgerrit | A change was merged to openstack/requirements: Work around atexit bug in python < 2.7.4 https://review.openstack.org/89382 | 02:36 |
| mattoliverau | ptoohill: So you have signed the aggreement? And are the contact details up to date, do they match your lauchpad account etc? | 02:36 |
| ptoohill | i cannot submit as Gerrit complains | 02:36 |
| ptoohill | yes, i have signed the agreement and all details match up | 02:37 |
| portante | clarkb: did you see the above on tox -e func for swift? | 02:37 |
| ptoohill | When attempting to submit contact changes i get server error | 02:37 |
| clarkb | portante no currently attemptibg vacation | 02:37 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack-dev/hacking: Updated from global requirements https://review.openstack.org/91220 | 02:37 |
| portante | run! | 02:37 |
| portante | and don't look back | 02:37 |
| portante | it can wait | 02:37 |
| ptoohill | "Cannot store contact information" | 02:38 |
| crc32 | it also says he already signed the ICLA. | 02:38 |
| ptoohill | ^ | 02:38 |
| mattoliverau | portante: Hmm, so the problem is gerrit details aren't correct and you can't update them then. | 02:38 |
| mattoliverau | ptoohill: ^^^ | 02:39 |
| ptoohill | thats what it seems like | 02:39 |
| mattoliverau | damn auto correct | 02:39 |
| mattoliverau | portante: sorry, wrong auto correct. | 02:39 |
| ptoohill | im not sure if theres another issue being hidden here somehow, but yes, git review complains that i need to submit contact info, but Gerrit wont let me | 02:39 |
| crc32 | mattoliverau: or they are correct but gerrit doesn't recognize a working config. | 02:39 |
| ptoohill | or that too ^ | 02:39 |
| portante | mattoliverau: no problem | 02:40 |
| ptoohill | My issue is nearly identical to this: https://ask.openstack.org/en/question/3413/icla-contributor-agreement-requires-current-contact-information-error-while-using-git-review/ | 02:41 |
| *** praneshp has joined #openstack-infra | 02:41 | |
| ptoohill | So, i assume, if Gerrit would take my attempts to add contact info the problem would be resolved. Unfortunately, Gerrit gives a server error | 02:42 |
| mattoliverau | ptoohill, crc32: In either case, I don't have admin rights... so can't poke around. So looks like we'll need a core to take a look. But I'll have a look around and see if the is anything else I can find we can try. | 02:43 |
| ptoohill | I do appreciate the help mattoliverau | 02:43 |
| *** yjiang has joined #openstack-infra | 02:43 | |
| crc32 | ok thank you. | 02:44 |
| crc32 | Will any core reviewer do? | 02:44 |
| *** Shrews has joined #openstack-infra | 02:46 | |
| *** sweston has joined #openstack-infra | 02:47 | |
| *** etoews has quit IRC | 02:51 | |
| *** zz_gondoi is now known as gondoi | 02:53 | |
| *** marun has quit IRC | 02:54 | |
| *** gyee has quit IRC | 02:55 | |
| *** marun has joined #openstack-infra | 02:55 | |
| *** matsuhashi has quit IRC | 02:56 | |
| *** changbl has joined #openstack-infra | 02:56 | |
| *** matsuhashi has joined #openstack-infra | 02:57 | |
| mattoliverau | crc32: I think so, so long as they have access to gerrit to maybe see a log or too to find out what happening and to hopefully fix it. | 02:57 |
| mattoliverau | gerrit was updated yesterday to a later version, so we could be experiencing a bug with the new version. | 02:58 |
| *** marun has quit IRC | 03:00 | |
| *** alexpilotti has quit IRC | 03:01 | |
| *** asalkeld has joined #openstack-infra | 03:02 | |
| *** gondoi is now known as zz_gondoi | 03:03 | |
| asalkeld | anyone know where the WIP button has gone from gerrit? nothing about it here -> https://wiki.openstack.org/wiki/GerritUpgrade | 03:03 |
| *** alexpilotti has joined #openstack-infra | 03:03 | |
| mattoliverau | asalkeld: its workflow -1 | 03:03 |
| asalkeld | that's not approved? | 03:03 |
| mattoliverau | thats workflow +1 | 03:04 |
| mattoliverau | there should be a label so you can't get it wrong | 03:04 |
| asalkeld | what I mean is approved == workflow +1, never approve == workflow -1 | 03:04 |
| *** alexpilotti has quit IRC | 03:04 | |
| asalkeld | not exactly the same as wip | 03:05 |
| asalkeld | nothing telling me someone is working on it | 03:05 |
| *** zz_gondoi is now known as gondoi | 03:05 | |
| ptoohill | mattoliverau, Issue has been resolved, made my first commit! Thanks again | 03:06 |
| mattoliverau | asalkeld: yeah, it isn't the best, so now a big red X in workflow means don't review its a WIP. | 03:07 |
| *** gondoi is now known as zz_gondoi | 03:07 | |
| mattoliverau | ptoohill: excelent, nice work! | 03:07 |
| asalkeld | grumble, grumble ... | 03:07 |
| asalkeld | thanks | 03:07 |
| *** amcrn has joined #openstack-infra | 03:07 | |
| *** Ryan_Lane has joined #openstack-infra | 03:07 | |
| *** talluri has joined #openstack-infra | 03:08 | |
| mattoliverau | asalkeld: yeah, new gerrit gives us an API so hopefully we can do something with that. | 03:08 |
| mattoliverau | in the meantime grumble away :P | 03:08 |
| clarkb | you can search on it too | 03:08 |
| asalkeld | lol so now we have create our own ui | 03:08 |
| clarkb | workflow>=0 to ignore wip | 03:08 |
| asalkeld | thanks - later (sorry for the grumbling) | 03:10 |
| *** asalkeld has left #openstack-infra | 03:10 | |
| *** nati_ueno has quit IRC | 03:10 | |
| *** marun has joined #openstack-infra | 03:11 | |
| *** nati_ueno has joined #openstack-infra | 03:11 | |
| mattoliverau | clarkb: yeah good point :) oh and thanks for helping in channel even though you should be vacationing! | 03:12 |
| *** crc32 has quit IRC | 03:13 | |
| openstackgerrit | A change was merged to openstack-infra/config: Bump check-rally-dsvm-{name} job timeout https://review.openstack.org/90630 | 03:14 |
| *** JayF has quit IRC | 03:20 | |
| *** JayF has joined #openstack-infra | 03:20 | |
| *** manishg has joined #openstack-infra | 03:20 | |
| *** chandan_kumar has joined #openstack-infra | 03:21 | |
| *** manishg has quit IRC | 03:25 | |
| *** manishg has joined #openstack-infra | 03:26 | |
| *** ianw has quit IRC | 03:27 | |
| *** ianw has joined #openstack-infra | 03:28 | |
| *** pcrews_ has quit IRC | 03:28 | |
| *** resker has joined #openstack-infra | 03:28 | |
| fungi | ptoohill: was the error you were getting trying to update your contact information covered by the answer at https://wiki.openstack.org/wiki/CLA-FAQ#When_trying_to_sign_the_new_ICLA_and_include_contact_information.2C_why_am_I.27m_getting_an_error_message_saying_that_my_E-mail_address_doesn.27t_correspond_to_a_Foundation_membership.3F | 03:29 |
| ptoohill | fungi, i didnt see this one, nor try anything else. i just used another account :/ | 03:30 |
| *** kashyap has quit IRC | 03:30 | |
| *** manishg has quit IRC | 03:30 | |
| fungi | ptoohill: ahh, well the faq there should hopefully cover most common cla/contact-info related errors | 03:30 |
| *** manishg has joined #openstack-infra | 03:31 | |
| ptoohill | fungi: ill keep that one in mind if error comes up again, but i wanted to actually use this new email anyhow, thank you though! | 03:31 |
| *** chandan_kumar has quit IRC | 03:32 | |
| *** esker has quit IRC | 03:32 | |
| fungi | in other news, there are devstack-f20 nodepool nodes now, and the devstack-precise and bare-centos6 node images i updated didn't break, so we should be all set | 03:32 |
| *** kashyap has joined #openstack-infra | 03:35 | |
| *** etoews has joined #openstack-infra | 03:36 | |
| *** manishg_ has joined #openstack-infra | 03:36 | |
| *** manishg has quit IRC | 03:36 | |
| *** freyes has quit IRC | 03:36 | |
| HenryG | Hi. What bug can I use to reverify https://review.openstack.org/90456 ? | 03:38 |
| HenryG | Here is the failure: http://logs.openstack.org/56/90456/3/gate/gate-neutron-python26/a7718f5/console.html | 03:38 |
| *** etoews has quit IRC | 03:40 | |
| *** manishg_ has quit IRC | 03:40 | |
| *** ArxCruz_ has quit IRC | 03:40 | |
| *** manishg has joined #openstack-infra | 03:41 | |
| *** jhesketh_ has joined #openstack-infra | 03:44 | |
| *** manishg has quit IRC | 03:46 | |
| *** manishg has joined #openstack-infra | 03:46 | |
| *** nati_ueno has quit IRC | 03:50 | |
| *** zz_gondoi is now known as gondoi | 03:50 | |
| *** manishg has quit IRC | 03:51 | |
| *** mfer has joined #openstack-infra | 03:51 | |
| *** chandan_kumar has joined #openstack-infra | 03:51 | |
| *** gondoi is now known as zz_gondoi | 03:51 | |
| *** manishg has joined #openstack-infra | 03:52 | |
| *** Ryan_Lane has quit IRC | 03:54 | |
| *** zz_gondoi is now known as gondoi | 03:54 | |
| *** Ryan_Lane has joined #openstack-infra | 03:55 | |
| *** mfer has quit IRC | 03:56 | |
| *** manishg has quit IRC | 03:56 | |
| *** manishg has joined #openstack-infra | 03:57 | |
| *** manishg has quit IRC | 04:01 | |
| *** manishg has joined #openstack-infra | 04:02 | |
| *** gondoi is now known as zz_gondoi | 04:04 | |
| *** manishg has quit IRC | 04:07 | |
| *** manishg has joined #openstack-infra | 04:07 | |
| *** mattymo|home has joined #openstack-infra | 04:09 | |
| *** matsuhashi has quit IRC | 04:10 | |
| *** manishg has quit IRC | 04:12 | |
| *** manishg has joined #openstack-infra | 04:12 | |
| *** praneshp has quit IRC | 04:17 | |
| *** praneshp_ has joined #openstack-infra | 04:17 | |
| *** manishg has quit IRC | 04:17 | |
| *** manishg has joined #openstack-infra | 04:18 | |
| *** manishg has quit IRC | 04:22 | |
| *** manishg has joined #openstack-infra | 04:23 | |
| *** nati_ueno has joined #openstack-infra | 04:24 | |
| *** yfried has quit IRC | 04:25 | |
| *** nosnos has quit IRC | 04:26 | |
| *** manishg has quit IRC | 04:27 | |
| *** Longgeek has joined #openstack-infra | 04:27 | |
| *** manishg has joined #openstack-infra | 04:28 | |
| *** manishg has quit IRC | 04:32 | |
| *** manishg has joined #openstack-infra | 04:33 | |
| *** manishg_ has joined #openstack-infra | 04:36 | |
| *** manishg has quit IRC | 04:38 | |
| *** manishg_ has quit IRC | 04:40 | |
| *** manishg has joined #openstack-infra | 04:41 | |
| tchaypo | dstufft: so all the tests pass now. | 04:43 |
| tchaypo | I'm not entirely certain if the message body that is being passed back is something useful though. | 04:43 |
| *** nati_ueno has quit IRC | 04:45 | |
| *** manishg_ has joined #openstack-infra | 04:46 | |
| *** manishg has quit IRC | 04:46 | |
| *** nati_ueno has joined #openstack-infra | 04:47 | |
| tchaypo | but since I don'tt hink that message ever gets utilised anywhere I don't think it matters | 04:48 |
| *** manishg_ has quit IRC | 04:50 | |
| *** alexpilotti has joined #openstack-infra | 04:51 | |
| *** manishg has joined #openstack-infra | 04:51 | |
| *** manishg has quit IRC | 04:56 | |
| *** manishg has joined #openstack-infra | 04:56 | |
| *** alexpilotti has quit IRC | 04:56 | |
| mattoliverau | fungi: thanks for the FAQ link, I'll add it to my list of bookmarks for future reference :) | 04:56 |
| *** manishg has quit IRC | 05:01 | |
| *** manishg has joined #openstack-infra | 05:02 | |
| *** ildikov has quit IRC | 05:03 | |
| *** matsuhashi has joined #openstack-infra | 05:04 | |
| *** manishg has quit IRC | 05:06 | |
| *** manishg has joined #openstack-infra | 05:07 | |
| *** UtahDave has joined #openstack-infra | 05:08 | |
| *** morganfainberg is now known as morganfainberg_Z | 05:10 | |
| *** manishg has quit IRC | 05:11 | |
| *** manishg has joined #openstack-infra | 05:12 | |
| *** nosnos has joined #openstack-infra | 05:16 | |
| *** manishg has quit IRC | 05:17 | |
| *** kashyap has quit IRC | 05:17 | |
| *** manishg has joined #openstack-infra | 05:17 | |
| *** kashyap has joined #openstack-infra | 05:17 | |
| ianw | fungi: still around? | 05:19 |
| *** zhiyan is now known as zhiyan_ | 05:20 | |
| *** manishg has quit IRC | 05:21 | |
| *** nosnos has quit IRC | 05:22 | |
| *** manishg has joined #openstack-infra | 05:22 | |
| openstackgerrit | A change was merged to openstack/requirements: Limit psutil to <2.0.0 https://review.openstack.org/81373 | 05:23 |
| *** nosnos has joined #openstack-infra | 05:23 | |
| *** AaronGreen has joined #openstack-infra | 05:25 | |
| *** manishg has quit IRC | 05:27 | |
| *** manishg has joined #openstack-infra | 05:28 | |
| *** AaronGr has quit IRC | 05:29 | |
| *** matsuhashi has quit IRC | 05:30 | |
| *** matsuhashi has joined #openstack-infra | 05:30 | |
| *** manishg has quit IRC | 05:32 | |
| *** manishg has joined #openstack-infra | 05:33 | |
| *** Ryan_Lane has quit IRC | 05:33 | |
| *** Ryan_Lane has joined #openstack-infra | 05:34 | |
| *** matsuhashi has quit IRC | 05:36 | |
| *** matsuhashi has joined #openstack-infra | 05:36 | |
| *** etoews has joined #openstack-infra | 05:37 | |
| *** manishg has quit IRC | 05:38 | |
| *** manishg has joined #openstack-infra | 05:38 | |
| *** manishg has quit IRC | 05:43 | |
| *** yfried has joined #openstack-infra | 05:43 | |
| *** manishg has joined #openstack-infra | 05:44 | |
| *** etoews has quit IRC | 05:45 | |
| *** resker has quit IRC | 05:48 | |
| *** UtahDave has quit IRC | 05:49 | |
| *** manishg has quit IRC | 05:49 | |
| *** manishg has joined #openstack-infra | 05:49 | |
| *** ildikov has joined #openstack-infra | 05:49 | |
| *** manishg has quit IRC | 05:53 | |
| *** amcrn has quit IRC | 05:54 | |
| *** manishg has joined #openstack-infra | 05:54 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/requirements: Update openstack-doc-tools to 0.13 to fix api-site https://review.openstack.org/90746 | 05:57 |
| *** manishg has quit IRC | 05:59 | |
| *** zul has quit IRC | 05:59 | |
| *** manishg has joined #openstack-infra | 06:00 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Publish translated localized API documents https://review.openstack.org/91132 | 06:01 |
| *** manishg has quit IRC | 06:04 | |
| *** manishg has joined #openstack-infra | 06:05 | |
| *** rcarrill` has joined #openstack-infra | 06:05 | |
| *** rcarrillocruz has quit IRC | 06:07 | |
| *** melwitt has quit IRC | 06:08 | |
| yfried | anyone here familiar with logstash? I have some questions about these filters https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb | 06:10 |
| *** WaltNik has joined #openstack-infra | 06:10 | |
| *** manishg has quit IRC | 06:10 | |
| *** manishg_ has joined #openstack-infra | 06:10 | |
| *** afazekas has joined #openstack-infra | 06:14 | |
| *** zul has joined #openstack-infra | 06:14 | |
| yfried | mattoliverau: ^ | 06:14 |
| *** manishg_ has quit IRC | 06:15 | |
| *** manishg has joined #openstack-infra | 06:16 | |
| *** medieval1 has quit IRC | 06:17 | |
| *** viktors|afk is now known as viktors | 06:19 | |
| *** Ryan_Lane has quit IRC | 06:19 | |
| *** medieval1 has joined #openstack-infra | 06:20 | |
| *** Ryan_Lane has joined #openstack-infra | 06:20 | |
| *** manishg has quit IRC | 06:20 | |
| mattoliverau | yfried: Not really up to speed with logstash, I plan to familarise myself with it when I get the chance as quite a few questions seem to be popping up about it. Feel free to ask, if I can't answer then they will be in scroll back for someone too when someone who knows something comes online. | 06:21 |
| *** manishg has joined #openstack-infra | 06:22 | |
| *** jcoufal has joined #openstack-infra | 06:24 | |
| *** manishg has quit IRC | 06:26 | |
| *** e0ne has joined #openstack-infra | 06:26 | |
| yfried | q1: what is the multi line filter for? shouldn't each line of log be parsed independantly? | 06:26 |
| *** manishg has joined #openstack-infra | 06:27 | |
| *** jlibosva has joined #openstack-infra | 06:29 | |
| *** ArxCruz has joined #openstack-infra | 06:30 | |
| *** _nadya_ has joined #openstack-infra | 06:31 | |
| *** manishg has quit IRC | 06:32 | |
| *** manishg has joined #openstack-infra | 06:32 | |
| *** ArxCruz has quit IRC | 06:35 | |
| *** manishg has quit IRC | 06:36 | |
| *** jcoufal has quit IRC | 06:37 | |
| *** jcoufal has joined #openstack-infra | 06:37 | |
| yfried | I assume it is to catch traceback messages, so that anything without timestamp is part of previous trace, but that's not how logs are built | 06:37 |
| *** manishg has joined #openstack-infra | 06:37 | |
| clarkb | yfried that is why. I dont understand your statement about how they are built | 06:39 |
| *** alexpilotti has joined #openstack-infra | 06:39 | |
| yfried | clarkb: http://paste.openstack.org/show/77885/ | 06:40 |
| yfried | clarkb: look for "self." to find a tracebak | 06:40 |
| clarkb | well its not just tracebacks | 06:41 |
| yfried | clarkb: what do you mean? | 06:41 |
| clarkb | its any line without a timestamp belongs to the previous line | 06:41 |
| yfried | clarkb: ok | 06:41 |
| mattoliverau | yfried: good question.. while clark it taking a look, I'm going to take the chance to drive to the otherside of town for dinner with the inlaws. | 06:41 |
| mattoliverau | night all, if I don't jump back online post dinner | 06:41 |
| yfried | mattoliverau: tnx | 06:41 |
| *** manishg has quit IRC | 06:42 | |
| clarkb | yfried: there are other occurrences of multine logs | 06:42 |
| *** manishg has joined #openstack-infra | 06:42 | |
| yfried | clarkb: but traceback in particular DO have timestamps. how would the match the previous line? | 06:42 |
| yfried | **they match | 06:43 |
| clarkb | tracebacks with the exception level are matched on log level | 06:43 |
| yfried | clarkb: what do you mean? | 06:43 |
| *** mkoderer has joined #openstack-infra | 06:44 | |
| clarkb | when they have a timestamp they have a different log level. that log level is used to collapse them together | 06:44 |
| *** alexpilotti has quit IRC | 06:45 | |
| yfried | clarkb: you mean loglevel = "TRACE" ? | 06:45 |
| *** e0ne has quit IRC | 06:45 | |
| clarkb | yes | 06:45 |
| *** manishg has quit IRC | 06:47 | |
| yfried | clarkb: this? https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb#L40 | 06:47 |
| *** manishg has joined #openstack-infra | 06:47 | |
| clarkb | yes | 06:48 |
| *** flaper87|afk is now known as flaper87 | 06:49 | |
| yfried | clarkb: does it mean all TRACE msgs are a single muiltiline, or do we assume all lines of a single trace have the exact same timestamp and NUMBER and each pare (TS:NUM) creates a new mulline match? | 06:49 |
| *** esker has joined #openstack-infra | 06:49 | |
| clarkb | they dont have the same timestamp and number. that is a regex | 06:50 |
| clarkb | it means all trace lines that havent been broken by a regex that doesnt match are a single multiline | 06:50 |
| *** flaper87 is now known as flaper87|afk | 06:51 | |
| yfried | clarkb: oh. so if for some reasone, a non-match line appears in the middle of a trace - a new trace is created by logstash? | 06:51 |
| yfried | clarkb: tnx | 06:51 |
| clarkb | yes anything that doesnt match breaks the event into a new event | 06:52 |
| yfried | clarkb: tnx | 06:52 |
| yfried | clarkb: new question: | 06:52 |
| yfried | https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb#L117 | 06:52 |
| yfried | what's the meaning of those mutate filters? | 06:52 |
| *** manishg has quit IRC | 06:52 | |
| yfried | clarkb: why do they remove a field? | 06:52 |
| *** manishg has joined #openstack-infra | 06:53 | |
| clarkb | the first one replaces the message with the constructed message it doesnt remove it completely | 06:54 |
| *** esker has quit IRC | 06:54 | |
| clarkb | the second one removes the now redundant fields | 06:54 |
| *** andreykurilin_ has joined #openstack-infra | 06:55 | |
| *** marun has quit IRC | 06:55 | |
| yfried | clarkb: why is logdate redundant? | 06:55 |
| clarkb | because it is in the timestamp field | 06:56 |
| clarkb | the date filter does that | 06:56 |
| yfried | clarkb: ok | 06:56 |
| yfried | clarkb: this is nit: since we now have logmsg and msg, why move logmsg into msg and delete logmsg? why not simply delete msg? | 06:57 |
| *** manishg has quit IRC | 06:57 | |
| yfried | clarkb: or might there by events without logmsg? | 06:57 |
| clarkb | because logstash convention is that the message is in message | 06:58 |
| yfried | clarkb: :) | 06:58 |
| yfried | clarkb: tnx | 06:58 |
| *** manishg has joined #openstack-infra | 06:58 | |
| yfried | clarkb: you helped me alot. will you be online for the following hours, or is it the end of your day? | 06:59 |
| clarkb | this is end of day | 06:59 |
| clarkb | and technically out this week soprobably not much irc until next week | 07:00 |
| yfried | clarkb: that's too bad. you seem like to only logstash expert around | 07:00 |
| yfried | clarkb: or do you know of anyone else? | 07:00 |
| yfried | who might also be in Europe timezone? | 07:01 |
| clarkb | I'm sure others can sort it out | 07:01 |
| clarkb | its mostly just regexes | 07:01 |
| yfried | clarkb: about the multiline - shouldn't no-timestamped line somehow get detailes from the previously stamped line (it's usually a continuation of a debug msg)? | 07:03 |
| *** manishg_ has joined #openstack-infra | 07:03 | |
| *** manishg has quit IRC | 07:03 | |
| *** flaper87|afk is now known as flaper87 | 07:03 | |
| clarkb | yes that is what multiline does | 07:04 |
| yfried | clarkb: ok. thnx | 07:04 |
| *** ArxCruz has joined #openstack-infra | 07:04 | |
| yfried | clarkb: why add "recieved_at"? isn't it duplication of timestamp? | 07:06 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack-infra/config: Import oslo.db repository https://review.openstack.org/78955 | 07:07 |
| *** manishg_ has quit IRC | 07:07 | |
| *** matsuhashi has quit IRC | 07:08 | |
| clarkb | no received at is when logstash received the log line | 07:08 |
| *** manishg has joined #openstack-infra | 07:08 | |
| clarkb | it gauges processing delay | 07:09 |
| *** pblaho has joined #openstack-infra | 07:10 | |
| *** matsuhashi has joined #openstack-infra | 07:11 | |
| *** manishg has quit IRC | 07:13 | |
| *** manishg has joined #openstack-infra | 07:14 | |
| yfried | clarkb: I'm trying to ship this file: http://paste.openstack.org/show/77881/ | 07:15 |
| yfried | clarkb: using this http://paste.openstack.org/show/77894 | 07:16 |
| yfried | clarkb: all 33 lines fail to ship, unless I completely disable the filter section | 07:17 |
| *** Daviey has joined #openstack-infra | 07:17 | |
| *** matsuhashi has quit IRC | 07:17 | |
| *** ekarlso has joined #openstack-infra | 07:17 | |
| *** matsuhashi has joined #openstack-infra | 07:17 | |
| *** _nadya_ has quit IRC | 07:18 | |
| yfried | clarkb: these are the logs of the 2 tries: | 07:18 |
| yfried | clarkb: with filteres - http://paste.openstack.org/show/77874/ | 07:18 |
| *** manishg has quit IRC | 07:18 | |
| yfried | clarkb: without filter - http://paste.openstack.org/show/77875/ | 07:18 |
| *** wenlock has quit IRC | 07:19 | |
| *** manishg has joined #openstack-infra | 07:19 | |
| *** talluri has quit IRC | 07:21 | |
| clarkb | yfried: sorry internet connection is terrible I wont be able to debug that | 07:21 |
| *** _nadya_ has joined #openstack-infra | 07:21 | |
| *** manishg_ has joined #openstack-infra | 07:24 | |
| *** manishg has quit IRC | 07:24 | |
| *** _nadya_ has quit IRC | 07:24 | |
| *** mattymo|home has quit IRC | 07:25 | |
| yfried | clarkb: could you think of a way to catch all filttered out events? | 07:27 |
| *** manishg_ has quit IRC | 07:28 | |
| clarkb | there shouldnt be any filtered out as you dont have any drops | 07:28 |
| *** marun has joined #openstack-infra | 07:29 | |
| clarkb | I would test with the stdout output though | 07:29 |
| *** manishg has joined #openstack-infra | 07:29 | |
| *** nati_ueno has quit IRC | 07:29 | |
| *** _nadya_ has joined #openstack-infra | 07:30 | |
| clarkb | the last bit of multiline data wont flush until a non matching line arrives or a timeout is hit though | 07:30 |
| *** amcrn has joined #openstack-infra | 07:30 | |
| *** mattymo|home has joined #openstack-infra | 07:30 | |
| *** andreykurilin_ has quit IRC | 07:30 | |
| *** manishg has quit IRC | 07:34 | |
| *** manishg has joined #openstack-infra | 07:35 | |
| *** praneshp_ has quit IRC | 07:38 | |
| *** manishg has quit IRC | 07:39 | |
| *** manishg has joined #openstack-infra | 07:40 | |
| *** etoews has joined #openstack-infra | 07:42 | |
| *** hashar has joined #openstack-infra | 07:43 | |
| yfried | clarkb: I disabled the multiline. now I only have 6 (instead of 33) events, and all are marked with _grokparsefaiure tag and their msg is replaced by "%{logmessage}" | 07:45 |
| *** manishg has quit IRC | 07:45 | |
| *** manishg_ has joined #openstack-infra | 07:45 | |
| *** romcheg1 has joined #openstack-infra | 07:45 | |
| romcheg1 | Guys, is anyone around who can help me with the stuck review? | 07:46 |
| romcheg1 | https://review.openstack.org/#/c/86021/ it was approved but gate jobs did not start | 07:47 |
| *** manishg_ has quit IRC | 07:49 | |
| *** etoews has quit IRC | 07:50 | |
| *** manishg has joined #openstack-infra | 07:50 | |
| *** esker has joined #openstack-infra | 07:50 | |
| *** ArxCruz has quit IRC | 07:50 | |
| *** Shivraam has joined #openstack-infra | 07:51 | |
| *** medieval1 has quit IRC | 07:51 | |
| yfried | clarkb: maybe I need the recieving server configured to catch multiline? (input codec)? | 07:51 |
| *** e0ne has joined #openstack-infra | 07:52 | |
| *** talluri has joined #openstack-infra | 07:52 | |
| *** esker has quit IRC | 07:54 | |
| *** talluri_ has joined #openstack-infra | 07:55 | |
| *** manishg has quit IRC | 07:55 | |
| *** manishg_ has joined #openstack-infra | 07:55 | |
| *** zigo_ is now known as zigo | 07:55 | |
| *** talluri has quit IRC | 07:55 | |
| *** medieval1 has joined #openstack-infra | 07:56 | |
| *** jpich has joined #openstack-infra | 07:57 | |
| *** manishg_ has quit IRC | 07:59 | |
| *** manishg has joined #openstack-infra | 08:00 | |
| *** Shivraam has quit IRC | 08:02 | |
| *** ildikov_ has joined #openstack-infra | 08:04 | |
| *** cody-somerville has quit IRC | 08:04 | |
| *** ildikov has quit IRC | 08:04 | |
| *** manishg has quit IRC | 08:04 | |
| *** manishg has joined #openstack-infra | 08:05 | |
| *** e0ne has quit IRC | 08:05 | |
| *** darraghb has joined #openstack-infra | 08:08 | |
| openstackgerrit | Carl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality https://review.openstack.org/91051 | 08:09 |
| *** manishg has quit IRC | 08:09 | |
| *** manishg has joined #openstack-infra | 08:10 | |
| *** e0ne has joined #openstack-infra | 08:13 | |
| *** manishg has quit IRC | 08:15 | |
| *** manishg has joined #openstack-infra | 08:15 | |
| *** mrmartin has joined #openstack-infra | 08:16 | |
| *** ihrachyshka has joined #openstack-infra | 08:17 | |
| *** cody-somerville has joined #openstack-infra | 08:19 | |
| *** cody-somerville has quit IRC | 08:19 | |
| *** cody-somerville has joined #openstack-infra | 08:19 | |
| *** manishg has quit IRC | 08:19 | |
| *** manishg has joined #openstack-infra | 08:20 | |
| *** derekh has joined #openstack-infra | 08:21 | |
| openstackgerrit | Benjamin Staffin proposed a change to openstack-infra/jenkins-job-builder: Tiny spelling fix: YMAL -> YAML https://review.openstack.org/91291 | 08:24 |
| *** manishg_ has joined #openstack-infra | 08:25 | |
| *** manishg has quit IRC | 08:25 | |
| *** mrmartin has quit IRC | 08:28 | |
| *** manishg_ has quit IRC | 08:30 | |
| *** manishg has joined #openstack-infra | 08:30 | |
| *** jooools has joined #openstack-infra | 08:31 | |
| openstackgerrit | Carl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality https://review.openstack.org/91051 | 08:34 |
| *** manishg has quit IRC | 08:35 | |
| *** manishg has joined #openstack-infra | 08:36 | |
| *** katyafervent is now known as katyafervent_awa | 08:37 | |
| *** manishg has quit IRC | 08:40 | |
| openstackgerrit | Nikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge https://review.openstack.org/91296 | 08:40 |
| openstackgerrit | Carl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality https://review.openstack.org/91051 | 08:41 |
| *** manishg has joined #openstack-infra | 08:42 | |
| *** katyafervent_awa is now known as katyafervent | 08:45 | |
| *** ArxCruz has joined #openstack-infra | 08:46 | |
| *** etoews has joined #openstack-infra | 08:46 | |
| *** manishg has quit IRC | 08:47 | |
| *** manishg has joined #openstack-infra | 08:47 | |
| *** romcheg1 has quit IRC | 08:49 | |
| *** ArxCruz has quit IRC | 08:50 | |
| *** ociuhandu has quit IRC | 08:51 | |
| *** etoews has quit IRC | 08:51 | |
| *** manishg has quit IRC | 08:52 | |
| *** jhesketh_ has quit IRC | 08:52 | |
| *** manishg has joined #openstack-infra | 08:52 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: TimeLine events support https://review.openstack.org/88405 | 08:55 |
| *** proffalken has quit IRC | 08:57 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Introducing the event timeline https://review.openstack.org/87959 | 08:57 |
| *** yjiang has quit IRC | 08:57 | |
| *** manishg has quit IRC | 08:57 | |
| *** manishg has joined #openstack-infra | 08:58 | |
| *** Ryan_Lane has quit IRC | 09:01 | |
| *** manishg has quit IRC | 09:02 | |
| *** manishg has joined #openstack-infra | 09:03 | |
| openstackgerrit | Nikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge https://review.openstack.org/91296 | 09:06 |
| *** jhesketh_ has joined #openstack-infra | 09:07 | |
| *** manishg has quit IRC | 09:07 | |
| *** manishg has joined #openstack-infra | 09:07 | |
| *** manishg_ has joined #openstack-infra | 09:12 | |
| *** manishg has quit IRC | 09:12 | |
| rossella_s | Hello all! I am having problems accessing review.openstack.org . I am based in Italy, if I use a VPN I can load the page without any problem. | 09:12 |
| BobBall | try restarting your browser session rossella_s | 09:12 |
| rossella_s | BobBall: I tried that and no joy | 09:13 |
| rossella_s | BobBall ssh is also not working...if I do ssh -p 29418 rossella-o@review.openstack.org gerrit ls-projects | 09:14 |
| rossella_s | it hangs | 09:14 |
| BobBall | that's a bigger problem :) | 09:14 |
| BobBall | hangs as in firewall eventually blocks it? | 09:14 |
| rossella_s | BobBall: yes :( | 09:14 |
| *** manishg has joined #openstack-infra | 09:14 | |
| BobBall | Did you know that the IP for review.openstack.org has changed? you might need to hange your firewall config | 09:14 |
| rossella_s | BobBall: yes I know the IP changed | 09:15 |
| BobBall | Also if you're using IPv6 I think some were having problems yesterday | 09:15 |
| BobBall | so try ipv4 | 09:15 |
| openstackgerrit | Nikolay Mahotkin proposed a change to openstack-infra/config: Add mistral-dashboard project to stackforge https://review.openstack.org/91296 | 09:15 |
| rossella_s | BobBall: I am using ipV4 | 09:16 |
| *** manishg_ has quit IRC | 09:16 | |
| BobBall | then I'm afraid I'm out of ideas :/ | 09:16 |
| BobBall | that's all I've heard of as being the cause | 09:17 |
| BobBall | My issues were the browser-restart one and the new IP being blocked by our corporate firewall | 09:17 |
| rossella_s | BobBall: my internet provider has a firewall, maybe that's where it's being blocked. How can I check that? | 09:18 |
| rossella_s | BobBall: thanks a lot for your patience! | 09:18 |
| rossella_s | BobBall: it's very unlikely that it's blocking it...anyway it's worth making sure | 09:18 |
| *** manishg_ has joined #openstack-infra | 09:19 | |
| *** manishg has quit IRC | 09:19 | |
| BobBall | If you've going straight to your ISP then it's very unlikely to be blocked there since it's an outgoing connection not an incoming one | 09:19 |
| BobBall | and, as it worked before, they aren't blocking the port. | 09:19 |
| BobBall | You'll probably have to wait for some of the infra guys who know what's changed to get more advice on this one :/ | 09:20 |
| BobBall | oh | 09:20 |
| BobBall | one more thought | 09:20 |
| BobBall | what does review.openstack.org resolve to? | 09:20 |
| BobBall | if you added it to /etc/hosts then that'll be the problem | 09:20 |
| rossella_s | resolves to 23.253.232.87 | 09:21 |
| BobBall | drat | 09:21 |
| BobBall | worth a shot | 09:21 |
| BobBall | that's the right IP | 09:21 |
| rossella_s | I already purged the know host file | 09:21 |
| rossella_s | BobBall: thanks a lot for your help! | 09:22 |
| *** manishg has joined #openstack-infra | 09:23 | |
| *** manishg_ has quit IRC | 09:23 | |
| *** manishg_ has joined #openstack-infra | 09:26 | |
| *** manishg has quit IRC | 09:27 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Move status stylesheets into css file https://review.openstack.org/84921 | 09:27 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Add in sparklines to status page pipelines https://review.openstack.org/84922 | 09:27 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular https://review.openstack.org/91310 | 09:27 |
| *** ominakov has joined #openstack-infra | 09:29 | |
| *** manishg_ has quit IRC | 09:30 | |
| *** andreaf has joined #openstack-infra | 09:30 | |
| *** proffalken has joined #openstack-infra | 09:31 | |
| *** manishg has joined #openstack-infra | 09:31 | |
| *** manishg_ has joined #openstack-infra | 09:35 | |
| *** manishg has quit IRC | 09:35 | |
| *** manishg has joined #openstack-infra | 09:38 | |
| *** manishg_ has quit IRC | 09:39 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular https://review.openstack.org/91310 | 09:40 |
| *** manishg has quit IRC | 09:42 | |
| *** manishg has joined #openstack-infra | 09:43 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Fix zuul's app.js name https://review.openstack.org/91314 | 09:43 |
| *** che-arne has joined #openstack-infra | 09:43 | |
| *** manishg has quit IRC | 09:47 | |
| *** manishg has joined #openstack-infra | 09:48 | |
| *** rcarrillocruz has joined #openstack-infra | 09:49 | |
| *** etoews has joined #openstack-infra | 09:50 | |
| *** esker has joined #openstack-infra | 09:50 | |
| *** rcarrill` has quit IRC | 09:50 | |
| *** yaguang has quit IRC | 09:51 | |
| *** manishg has quit IRC | 09:52 | |
| *** manishg_ has joined #openstack-infra | 09:52 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Fix up fetching jquery.visibility https://review.openstack.org/91316 | 09:53 |
| *** etoews has quit IRC | 09:54 | |
| *** esker has quit IRC | 09:54 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Move status stylesheets into css file https://review.openstack.org/84921 | 09:56 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Fix up fetching jquery.visibility https://review.openstack.org/91316 | 09:56 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Add in sparklines to status page pipelines https://review.openstack.org/84922 | 09:56 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Make zuul javascript app more modular https://review.openstack.org/91310 | 09:56 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Fix pep8 issue with test_scheduler https://review.openstack.org/91317 | 09:56 |
| *** manishg_ has quit IRC | 09:56 | |
| *** manishg has joined #openstack-infra | 09:57 | |
| jhesketh_ | is anybody who was involved in the gerrit upgrade around? | 09:57 |
| jhesketh_ | fungi: ping | 09:58 |
| *** manishg has quit IRC | 10:01 | |
| *** manishg has joined #openstack-infra | 10:03 | |
| *** manishg_ has joined #openstack-infra | 10:07 | |
| *** manishg has quit IRC | 10:08 | |
| *** rcarrill` has joined #openstack-infra | 10:09 | |
| *** alexpilotti has joined #openstack-infra | 10:10 | |
| *** manishg_ has quit IRC | 10:11 | |
| *** rcarrillocruz has quit IRC | 10:11 | |
| *** manishg has joined #openstack-infra | 10:12 | |
| BobBall | don't think so jhesketh_ - they will all be sleeping | 10:13 |
| jhesketh_ | figured, all good :-) | 10:13 |
| jhesketh_ | th | 10:13 |
| jhesketh_ | *thanks | 10:13 |
| BobBall | actually sdague may be around at some point | 10:13 |
| BobBall | depends on the level of involvement you care about :) | 10:14 |
| jhesketh_ | just curious to how a patch that fails pep8 got through the gate.. maybe we're not gating properly for example | 10:14 |
| jhesketh_ | it's okay, I'll grab some dinner | 10:14 |
| *** jp_at_hp has joined #openstack-infra | 10:14 | |
| BobBall | Curious indeed! | 10:15 |
| *** manishg has quit IRC | 10:16 | |
| *** dizquierdo has joined #openstack-infra | 10:17 | |
| *** manishg has joined #openstack-infra | 10:17 | |
| sdague | what's up? | 10:19 |
| sdague | jhesketh_: pointer? | 10:19 |
| BobBall | sdague: I think it was a question about how the pep8 violation fixed by https://review.openstack.org/91317 got merged | 10:19 |
| BobBall | & whether the gate is working correctly (I guess it might have been force-pushed during the upgrade) | 10:20 |
| *** talluri_ has quit IRC | 10:20 | |
| *** nosnos has quit IRC | 10:21 | |
| *** manishg has quit IRC | 10:21 | |
| SergeyLukjanov | morning folks | 10:22 |
| *** manishg has joined #openstack-infra | 10:22 | |
| *** ociuhandu has joined #openstack-infra | 10:23 | |
| sdague | BobBall: looks like fungi merged it manually | 10:23 |
| sdague | the Verified vote is not from Jenkins | 10:23 |
| *** manishg has quit IRC | 10:27 | |
| *** manishg has joined #openstack-infra | 10:28 | |
| *** nosnos has joined #openstack-infra | 10:29 | |
| *** manishg has quit IRC | 10:32 | |
| *** manishg has joined #openstack-infra | 10:33 | |
| *** markmc has joined #openstack-infra | 10:34 | |
| *** manishg has quit IRC | 10:38 | |
| BobBall | there you fair enough | 10:38 |
| BobBall | -there you! | 10:38 |
| jhesketh_ | sdague: didn't realise you could merge it through manually | 10:38 |
| jhesketh_ | sdague: doesn't gerrit wait for --submit from zuul? | 10:38 |
| *** manishg has joined #openstack-infra | 10:38 | |
| *** etoews has joined #openstack-infra | 10:39 | |
| *** talluri has joined #openstack-infra | 10:41 | |
| sdague | jhesketh_: infra-core can direct merge | 10:41 |
| sdague | there are times when it's needed | 10:41 |
| sdague | normal humans can't | 10:41 |
| jhesketh_ | sdague: fair enough | 10:42 |
| jhesketh_ | do other cores for other projects have that? | 10:42 |
| *** manishg has quit IRC | 10:42 | |
| *** manishg has joined #openstack-infra | 10:43 | |
| *** etoews has quit IRC | 10:43 | |
| sdague | no | 10:44 |
| bookwar | can anyone help with gerrit authorization, here are the logs http://pastebin.com/print.php?i=0kqn07FZ and we tried hard to ensure that the key stored is the right one, can someone check it on the server side? | 10:47 |
| jhesketh_ | ah okay | 10:47 |
| jhesketh_ | do infra have verify +2 rights on other projects? | 10:47 |
| *** vkramskikh has joined #openstack-infra | 10:47 | |
| *** manishg has quit IRC | 10:47 | |
| *** manishg has joined #openstack-infra | 10:48 | |
| sdague | project bootstrappers has all the permissions | 10:48 |
| sdague | and infra core can add themselves to that group | 10:48 |
| sdague | when needed | 10:48 |
| sdague | they don't tend to keep themselves in there | 10:48 |
| jhesketh_ | righto | 10:50 |
| jhesketh_ | sdague: so how do the merges get triggered.. I thought you needed to send the submit command, or is gerrit instructed to simply look for the Verified+2? | 10:50 |
| *** openstackgerrit has quit IRC | 10:51 | |
| sdague | jhesketh_: that I'm not 100% sure on. fungi would know | 10:52 |
| *** manishg has quit IRC | 10:52 | |
| *** manishg has joined #openstack-infra | 10:52 | |
| jhesketh_ | sure, thanks for the clarifications though :-) | 10:52 |
| sdague | I think zuul is issuing the submit | 10:52 |
| sdague | once things are satisfied | 10:53 |
| *** vponomaryov has quit IRC | 10:53 | |
| jhesketh_ | hmm, not sure about that, looking at the commit that went through the merge was listed under Jeremy's user, not Jenkins' | 10:53 |
| *** ominakov has quit IRC | 10:54 | |
| *** miarmak has quit IRC | 10:54 | |
| *** achuprin has quit IRC | 10:54 | |
| *** vponomaryov has joined #openstack-infra | 10:56 | |
| *** yamahata has quit IRC | 10:56 | |
| *** ominakov has joined #openstack-infra | 10:56 | |
| *** miarmak has joined #openstack-infra | 10:56 | |
| *** manishg has quit IRC | 10:57 | |
| *** achuprin has joined #openstack-infra | 10:57 | |
| *** manishg has joined #openstack-infra | 10:57 | |
| *** manishg has quit IRC | 11:01 | |
| *** manishg has joined #openstack-infra | 11:02 | |
| *** manishg has quit IRC | 11:06 | |
| *** e0ne has quit IRC | 11:07 | |
| *** manishg has joined #openstack-infra | 11:07 | |
| *** manishg has quit IRC | 11:11 | |
| yfried | anyone here familiar with logstash? | 11:11 |
| *** manishg has joined #openstack-infra | 11:11 | |
| *** manishg has quit IRC | 11:16 | |
| *** manishg has joined #openstack-infra | 11:17 | |
| *** manishg has quit IRC | 11:22 | |
| *** adalbas has joined #openstack-infra | 11:23 | |
| *** manishg has joined #openstack-infra | 11:23 | |
| SergeyLukjanov | yfried, clarkb, but he's on vacation now | 11:27 |
| *** manishg has quit IRC | 11:28 | |
| yfried | SergeyLukjanov: yeah. we talked earlier. anyone else? | 11:28 |
| SergeyLukjanov | yfried, IIRC nope | 11:29 |
| *** manishg has joined #openstack-infra | 11:29 | |
| *** manishg has quit IRC | 11:33 | |
| *** manishg has joined #openstack-infra | 11:34 | |
| *** manishg has quit IRC | 11:39 | |
| *** mwagner_lap has quit IRC | 11:39 | |
| *** manishg has joined #openstack-infra | 11:40 | |
| *** manishg_ has joined #openstack-infra | 11:44 | |
| *** manishg has quit IRC | 11:44 | |
| *** rlandy has joined #openstack-infra | 11:46 | |
| *** manishg_ has quit IRC | 11:48 | |
| *** manishg has joined #openstack-infra | 11:50 | |
| *** esker has joined #openstack-infra | 11:50 | |
| *** ArxCruz has joined #openstack-infra | 11:50 | |
| *** manishg has quit IRC | 11:54 | |
| *** manishg has joined #openstack-infra | 11:54 | |
| *** esker has quit IRC | 11:55 | |
| *** Longgeek_ has joined #openstack-infra | 11:55 | |
| *** Longgeek has quit IRC | 11:58 | |
| *** manishg has quit IRC | 11:59 | |
| *** Longgeek_ has quit IRC | 11:59 | |
| *** e0ne has joined #openstack-infra | 12:00 | |
| *** manishg has joined #openstack-infra | 12:01 | |
| *** matsuhashi has quit IRC | 12:02 | |
| *** e0ne_ has joined #openstack-infra | 12:02 | |
| *** matsuhashi has joined #openstack-infra | 12:02 | |
| *** e0ne has quit IRC | 12:02 | |
| *** matsuhashi has quit IRC | 12:03 | |
| *** manishg has quit IRC | 12:05 | |
| *** manishg has joined #openstack-infra | 12:05 | |
| *** pdmars has joined #openstack-infra | 12:06 | |
| *** jgallard has joined #openstack-infra | 12:06 | |
| *** e0ne_ has quit IRC | 12:07 | |
| *** nosnos has quit IRC | 12:09 | |
| *** manishg has quit IRC | 12:10 | |
| *** manishg has joined #openstack-infra | 12:10 | |
| *** _nadya_ has quit IRC | 12:13 | |
| *** _nadya_ has joined #openstack-infra | 12:14 | |
| *** pblaho has quit IRC | 12:15 | |
| *** manishg has quit IRC | 12:15 | |
| *** manishg has joined #openstack-infra | 12:15 | |
| *** dprince has joined #openstack-infra | 12:16 | |
| *** skraynev_ has joined #openstack-infra | 12:16 | |
| *** skraynev_ has quit IRC | 12:16 | |
| *** sandywalsh has joined #openstack-infra | 12:17 | |
| *** manishg has quit IRC | 12:20 | |
| *** manishg has joined #openstack-infra | 12:21 | |
| tnurlygayanov | hi there! | 12:22 |
| tnurlygayanov | I have a question about the tarballs automated creation for the new releases | 12:23 |
| tnurlygayanov | we have the repository https://github.com/stackforge/mistral-extra | 12:23 |
| tnurlygayanov | and we have pushed the new tag 0.0.1 | 12:23 |
| tnurlygayanov | for the first stable version of Mistral | 12:24 |
| tnurlygayanov | but zuul creates tarball with incorrect version 0.0.0: http://tarballs.openstack.org/mistral-extra/mistral-extra-0.0.0.tar.gz | 12:25 |
| *** manishg has quit IRC | 12:25 | |
| tnurlygayanov | can anybody describe how we can create the tarbal with version 0.0.1 ? | 12:25 |
| tnurlygayanov | for other repositories of Mistral zuul creates correct tarball with version number 0.0.1 | 12:26 |
| *** skolekonov has joined #openstack-infra | 12:26 | |
| *** mwagner_lap has joined #openstack-infra | 12:27 | |
| *** manishg has joined #openstack-infra | 12:28 | |
| ArxCruz | anteaya: the powervc team fix the issue yesterday :) | 12:28 |
| *** rcarrill` is now known as rcarrillocruz | 12:28 | |
| *** aysyd has joined #openstack-infra | 12:31 | |
| *** manishg has quit IRC | 12:32 | |
| *** maxbit has quit IRC | 12:32 | |
| *** maxbit has joined #openstack-infra | 12:33 | |
| *** manishg has joined #openstack-infra | 12:33 | |
| *** maxbit has quit IRC | 12:33 | |
| *** maxbit has joined #openstack-infra | 12:36 | |
| *** weshay has joined #openstack-infra | 12:36 | |
| *** manishg has quit IRC | 12:37 | |
| *** maxbit has quit IRC | 12:37 | |
| *** manishg has joined #openstack-infra | 12:38 | |
| *** maxbit has joined #openstack-infra | 12:39 | |
| *** rfolco has joined #openstack-infra | 12:39 | |
| *** maxbit has quit IRC | 12:41 | |
| *** lcostantino has joined #openstack-infra | 12:41 | |
| *** maxbit has joined #openstack-infra | 12:42 | |
| *** manishg has quit IRC | 12:43 | |
| SergeyLukjanov | tnurlygayanov, hey, do you have link to the tarball job logs? | 12:44 |
| tnurlygayanov | yes | 12:44 |
| tnurlygayanov | SergeyLukjanov: https://jenkins03.openstack.org/job/mistral-extra-tarball/1/ | 12:44 |
| SergeyLukjanov | tnurlygayanov, it looks like err in setup.cfg at least | 12:45 |
| SergeyLukjanov | https://github.com/stackforge/mistral-extra/blob/master/setup.cfg#L3 | 12:45 |
| SergeyLukjanov | tnurlygayanov, and setup.py is incorrect - it doesn't call pbr https://github.com/stackforge/mistral-extra/blob/master/setup.py | 12:46 |
| tnurlygayanov | SergeyLukjanov: for Mistral repo we have the same error, but tarbal was successfully created. | 12:46 |
| tnurlygayanov | with correct version 0.0.1. | 12:46 |
| SergeyLukjanov | tnurlygayanov, the version is overriden by gerrit | 12:46 |
| tnurlygayanov | ok, we will fix all small issues with setup.cfg | 12:46 |
| SergeyLukjanov | tnurlygayanov, but setup.py should call pbr for it | 12:46 |
| SergeyLukjanov | tnurlygayanov, otherwise it'll not work | 12:46 |
| SergeyLukjanov | tnurlygayanov, https://github.com/openstack/sahara/blob/master/setup.py is the correct pbr init | 12:47 |
| SergeyLukjanov | tnurlygayanov, due to the https://github.com/stackforge/mistral-extra/blob/master/setup.py#L21 | 12:47 |
| SergeyLukjanov | setuptools.setup( | 12:47 |
| SergeyLukjanov | name="mistral-extra") | 12:47 |
| *** esker has joined #openstack-infra | 12:47 | |
| *** vkramskikh has left #openstack-infra | 12:47 | |
| *** rakhmerov has joined #openstack-infra | 12:47 | |
| SergeyLukjanov | version will be 0.0.0 anyway, there is no way to override it | 12:47 |
| *** julim has joined #openstack-infra | 12:49 | |
| tnurlygayanov | SergeyLukjanov: ok, thank you! | 12:49 |
| anteaya | ArxCruz: thank you! | 12:49 |
| anteaya | ArxCruz: I appreciate you talking to them and following up with me, thank you | 12:49 |
| ArxCruz | anteaya: you're welcome :) | 12:50 |
| *** mfer has joined #openstack-infra | 12:51 | |
| viktors | hello All! Folks, could you please look at the patch to config - https://review.openstack.org/#/c/78955/ (Import oslo.db repository) ? Thanks! | 12:52 |
| *** dkranz has joined #openstack-infra | 12:56 | |
| *** mbacchi has joined #openstack-infra | 12:59 | |
| *** eharney has joined #openstack-infra | 13:02 | |
| SergeyLukjanov | viktors, hey, lgtm | 13:05 |
| *** Ajaeger has joined #openstack-infra | 13:06 | |
| viktors | SergeyLukjanov: thanks a lot! | 13:06 |
| *** matty_dubs|gone is now known as matty_dubs | 13:07 | |
| *** smarcet has joined #openstack-infra | 13:13 | |
| *** jgallard has quit IRC | 13:17 | |
| *** mriedem has joined #openstack-infra | 13:19 | |
| *** thomasem has joined #openstack-infra | 13:19 | |
| *** mwagner_lap has quit IRC | 13:20 | |
| *** bknudson has joined #openstack-infra | 13:22 | |
| *** _nadya_ has quit IRC | 13:28 | |
| jeblair | jhesketh_: sdague: yes, that was manually merged bypassing gating during the maintenance to fix an issue with zuul itself (the removal of force-message in gerrit broke several of our pipelines). | 13:28 |
| anteaya | jeblair: does our gerrit use a self-signed cert? http://git.openstack.org/cgit/stackforge/gertty/tree/README.rst | 13:29 |
| jeblair | jhesketh_: sdague: fixing operational problems is about the only time we do that. | 13:29 |
| *** mengxd has joined #openstack-infra | 13:29 | |
| jeblair | anteaya: no, but review-dev does | 13:29 |
| anteaya | jeblair: k, thanks | 13:29 |
| *** signed8bit has joined #openstack-infra | 13:32 | |
| jeblair | is there a way i can make firefox look like firefox and not chrome? <sigh> | 13:33 |
| *** dkliban_afk is now known as dkliban | 13:33 | |
| *** yfried has quit IRC | 13:34 | |
| fungi | jhesketh_: sdague: what was the question? | 13:35 |
| *** afazekas has quit IRC | 13:35 | |
| jeblair | fungi: how did a pep8 violation in zuul merge | 13:35 |
| *** enykeev has quit IRC | 13:36 | |
| *** skolekonov has quit IRC | 13:36 | |
| jeblair | fungi: i answered them above ^ | 13:37 |
| *** mkerrin has quit IRC | 13:37 | |
| jhesketh_ | jeblair: okay, thanks | 13:37 |
| jhesketh_ | jeblair: so how does gerrit know when to actually do a merge? | 13:38 |
| fungi | jeblair: yep, i see that now | 13:38 |
| *** zz_gondoi is now known as gondoi | 13:38 | |
| fungi | jhesketh_: gerrit is instructed (by someone or something) to merge a change | 13:38 |
| fungi | jhesketh_: normally zuul instructs gerrit to merge a change once all jobs succeed on it | 13:39 |
| jhesketh_ | fungi: right, so is there some kind of force submit button that core infra people have? | 13:40 |
| fungi | jhesketh_: for unusual circumstances (like needing to coordinate merges of things which are blocked on outside alterations, such as a gerrit upgrade) we can force a change to merge with a button, yes | 13:40 |
| jeblair | we don't normally have extra acls, but we add ourselves to a group that has them when needed | 13:41 |
| fungi | jhesketh_: normally we don't "have" that button per se. we self-add to an administrative group which has elevated access within gerrit, and then we see the submit button, verify vote options and so on | 13:41 |
| jhesketh_ | right, makes sense :-) thanks | 13:41 |
| fungi | and then remove ourselves from said group once done | 13:41 |
| fungi | to avoid... "accidents" | 13:41 |
| jhesketh_ | heh, wise | 13:42 |
| *** mengxd has quit IRC | 13:42 | |
| *** dkehn_ has joined #openstack-infra | 13:43 | |
| fungi | and we really, really don't like doing it. because it can lead to issues like the pep8 failure you spotted (we were a bit rushed and not running tests manually locally when scrambling to get everything running again) | 13:43 |
| jhesketh_ | jeblair: so with that reporter issue, should the requirement to list reporters at all just be lifted? | 13:43 |
| *** plars has joined #openstack-infra | 13:44 | |
| jeblair | jhesketh_: actually the requirement to list reporters there was lucky and let us do that -- listing a reporter already says "i want you to send a report in this manner". that's all we want for experimental pipelines (no voting or anything). | 13:45 |
| jeblair | jhesketh_: but we need no arguments if all we're doing is sending a message. | 13:45 |
| *** dkehn_zzz has quit IRC | 13:45 | |
| *** dkehn_ has quit IRC | 13:46 | |
| jeblair | jhesketh_: we might want to find another way of doing that, because, yeah, the braces to make an empty dict are not exactly intuitive | 13:46 |
| StevenK | Could I get some kind souls to nudge https://review.openstack.org/82706 into landing? | 13:46 |
| *** dkehn_ has joined #openstack-infra | 13:46 | |
| *** dkehnx has quit IRC | 13:46 | |
| jeblair | jhesketh_: but the general approach of saying "i want gerrit to report success (full stop)" and "i want gerrit to report success and pass these other arguments " seems intuitive | 13:47 |
| *** dkehn_zzz has joined #openstack-infra | 13:48 | |
| plars | I heard that someone here might have experience with using jenkins open-id for sso along with python-jenkins for job management? We're experimenting with a similar setup and get_jobs() works if I pass username and api token for the password, but not job creation or reconfiguring | 13:49 |
| *** afazekas has joined #openstack-infra | 13:49 | |
| jeblair | i think gerritbot is dead | 13:49 |
| fungi | jeblair: are you looking into it? if not, i'll coax it back in | 13:50 |
| jeblair | fungi: on it | 13:50 |
| *** jgrimm has joined #openstack-infra | 13:50 | |
| jeblair | plars: you may need to check the authorization in jenkins to make sure your user has the right perms | 13:51 |
| jeblair | plars: all our job management is done with jenkins-job-builder, so you can check that for examples of how we use python-jenkins to interact | 13:51 |
| plars | jeblair: yeah, already checked that. The user has full administrative permissions. Even tried giving all the permissions specifically on top of that | 13:51 |
| anteaya | I have gertty running, and have subscribed to one repo stackforge/gertty | 13:52 |
| *** _nadya_ has joined #openstack-infra | 13:52 | |
| plars | jeblair: I also tried jjb, which I think is using python-jenkins underneath. I still get the 403 when trying to create a job | 13:52 |
| jeblair | plars: do you have xsrf protection enabled in jenkins? i think python-jenkins recently gained features to deal with that | 13:53 |
| jeblair | plars: but that's a possible cause | 13:53 |
| *** sdake_ has quit IRC | 13:53 | |
| fungi | bookwar: has anybody helped you with your authentication problem yet? the error message you pasted looks like you're using ssh-agent but haven't done an ssh-add on that key yet | 13:54 |
| jeblair | plars: also, sometimes in these situations, i add a print statement to python-jenkins to print out the output returned in the response body from jenkins to see if it has any helpful info | 13:54 |
| bookwar | fungi: is already silved, thanks, was issue on our side ) | 13:54 |
| *** freyes has joined #openstack-infra | 13:54 | |
| fungi | bookwar: okay, cool | 13:54 |
| jeblair | File "/usr/local/lib/python2.7/dist-packages/irc/client.py", line 892, in send_raw | 13:55 |
| jeblair | raise ServerNotConnectedError("Not connected.") | 13:55 |
| plars | jeblair: looks like it's turned off at the moment | 13:55 |
| *** dkehn_zzz is now known as dkehn | 13:56 | |
| jeblair | plars: i'm out of ideas off the top of my head; i'd try getting the output from the 403 then | 13:56 |
| *** openstackgerrit has joined #openstack-infra | 13:56 | |
| jeblair | anteaya: you should have an unreviewed change now :) | 13:57 |
| *** doug-fish has joined #openstack-infra | 13:59 | |
| anteaya | jeblair: thanks, it started using all my cpu so I had to shut it down, will start it again in a debugger to see if I can get some info next time | 14:00 |
| jeblair | anteaya: runing it with the '-d' option and tailing ~/.gertty.log might tell you what you need to know. | 14:01 |
| *** zns has joined #openstack-infra | 14:01 | |
| *** homeless has joined #openstack-infra | 14:02 | |
| anteaya | jeblair: I'll try that | 14:02 |
| openstackgerrit | Pierre Padrixe proposed a change to openstack/requirements: Update sphinxcontrib-pecanwsme to 0.8 https://review.openstack.org/91370 | 14:02 |
| anteaya | I was invoking with `python gertty/gertty.py` | 14:03 |
| *** dangers_away is now known as dangers | 14:03 | |
| anteaya | is that what I should be using? | 14:03 |
| openstackgerrit | A change was merged to stackforge/gertty: Fix some diff comment display errors https://review.openstack.org/91364 | 14:03 |
| jeblair | anteaya: yep. 'python gertty/gertty.py -d' will log debug output | 14:04 |
| anteaya | k | 14:04 |
| Ajaeger | infra team, previously annegentle and myself could both +2 and approve patches in the api projects like compute-api. We noticed yesterday that we still can +2 but not approve anymore. It works fine for the normal docu projects like openstack-manuals but not for these mixed projects like compute-api. | 14:04 |
| jhesketh_ | jeblair: (sorry, had to step away for a sec) right, but I meant what about having no reporters at all.. Should that be something we want possible? | 14:04 |
| jeblair | jhesketh_: oh, yes -- that's how we do the silent pipeline | 14:05 |
| jhesketh_ | yeah, I just checked the code | 14:05 |
| jhesketh_ | jeblair: right, I misinterpreted your commit msg | 14:05 |
| jeblair | Ajaeger: we probably missed an acl change, i'll look | 14:05 |
| fungi | Ajaeger: i'll take a look. you're trying to "workflow +1" them now righth? (approve has a new name) | 14:05 |
| jeblair | jhesketh_: sorry, heh, was under duress. :) | 14:05 |
| *** esker has quit IRC | 14:05 | |
| *** prad_ has joined #openstack-infra | 14:05 | |
| jeblair | fungi: nm, i'll let you handle it if you're on it | 14:06 |
| jhesketh_ | jeblair: heh, more likely my fault, it's after midnight here | 14:06 |
| Ajaeger | fungi: I have no workflow buttons at all for compute-api, they only exist for openstack-manuals etc | 14:06 |
| * hashar sends jhesketh_ to bed | 14:06 | |
| *** esker has joined #openstack-infra | 14:06 | |
| fungi | Ajaeger: ahh, i bet we missed changing the api-projects meta project acl | 14:06 |
| jhesketh_ | hashar: you fail to realise this is when I work best ;-) | 14:06 |
| jhesketh_ | although my previous statement disproves that :-P | 14:06 |
| *** yamahata has joined #openstack-infra | 14:06 | |
| hashar | jhesketh_: yeah have the same issue. But have to wake up at 7am no matter what happens (kid!) | 14:07 |
| jeblair | jhesketh_: haha :) | 14:07 |
| Ajaeger | fungi: thanks for checking | 14:07 |
| Ajaeger | hashar: 7am? That's late ;) | 14:07 |
| jhesketh_ | yeah, luckily I can choose my hours a bit :-) | 14:07 |
| jhesketh_ | but there's also a reason I don't often make it to the 5am infra meetings | 14:07 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories https://review.openstack.org/70373 | 14:08 |
| fungi | Ajaeger: yep, that was it... try again | 14:09 |
| openstackgerrit | Pierre Padrixe proposed a change to openstack/requirements: Update sphinxcontrib-pecanwsme to 0.8 https://review.openstack.org/91370 | 14:10 |
| Ajaeger | fungi: great, netconn-api and identity-api look fine now (no review open for compute-api ;). Thanks a lot! | 14:10 |
| fungi | Ajaeger: you're welcome. apologies for the oversight. we seem to do "special" things for acls on api projects | 14:11 |
| anteaya | jeblair: I see the sync number counts down | 14:11 |
| Ajaeger | fungi: yeah, api projects are special in that both project and docu cores have access. | 14:11 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories https://review.openstack.org/70373 | 14:12 |
| jeblair | anteaya: yes, it shows you how many background sync tasks it has going | 14:12 |
| hashar | would it make sense to have WIP -1 to stick when new patch sets are pushed ? | 14:12 |
| fungi | Ajaeger: in theory we could solve that much with group inclusion. looks like what we're doing special in this case is overriding the default state for "Forge Author Identity" and "Forge Committer Identity" on refs/* | 14:12 |
| * anteaya nods, continues to poke around | 14:13 | |
| jeblair | hashar, YorikSar suggested that too. unfortunately, it's a question with no right answer right now. half the time people want it to stack, the other half they don't. | 14:13 |
| boris-42 | fungi btw bumping timeout works woohoo=) | 14:13 |
| fungi | Ajaeger: might be for historical reasons no longer relevant... jeblair: do you happen to remember why they're set up that way? | 14:13 |
| jeblair | hashar, YorikSar: s/stack/stick/. | 14:14 |
| anteaya | jeblair: should I expect to see a link to the unreviewed change in the stackforge/gertty repo in my gertty view somewhere? | 14:14 |
| hashar | jeblair: not a big deal. I can just workflow -1 whenever I push a new one :] Thx! | 14:14 |
| jeblair | hashar, YorikSar: when you're continuously developing something, you might want it to stick; but when you say "oh, yeah, i'll fix this thing and push a new patchset" you don't. | 14:15 |
| jeblair | hashar, YorikSar: i think the best answer in the future might be to work on the work-in-progress plugin for gerrit | 14:15 |
| fungi | hashar: one downside to making workflow sticky for future patchsets is that it's not a toggle for the change, it's a toggle for multiple parties individually on the change. if a core reviewer adds workflow -1 to your change, you as the change owner will be unable to remove it. currently the solution is a new patchset or hunt them down and beg | 14:15 |
| anteaya | too bad we can't toggle that in preferences | 14:15 |
| anteaya | then individuals can choose | 14:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Import oslo.db repository https://review.openstack.org/78955 | 14:16 |
| anteaya | sticky -1 workflow or not | 14:16 |
| jeblair | hashar, YorikSar: especially if the plugin can define a magic ref, then git-review could push to 'refs/wip/foo'. | 14:16 |
| hashar | or we could use the draft system but the change is not public | 14:17 |
| jeblair | hashar, YorikSar: then it wouldn't be a two-stage thing (you can push a patchset directly into wip) | 14:17 |
| fungi | perhaps a compromise would be to make owner-provided workflow votes sticky and non-owner workflow votes clear on new patchsets | 14:17 |
| anteaya | fungi: that sounds like a good start | 14:17 |
| jeblair | hashar, YorikSar: but at the moment, there's a balance of interests in the workflow -1 system of doing wip -- if we make them sticky, then you can't remove someone else's wip vote. | 14:18 |
| fungi | though maybe still clear owner-added workflow +1 on new patchsets, since that could have unintended consequences for self-approving change owners | 14:18 |
| anteaya | so I see the title "unreviewed changes in stackforge/gertty" and the headers "Number" and "Subject" and the sync countdown and nothing else | 14:19 |
| jeblair | fungi: i'd rather have consistency so i can predict whether the vote will be cleared or not. i think the best solution is to actually allow the author to express his or her intent when uploading | 14:19 |
| jeblair | anteaya: i merged the change while you weren't running it. there are no open changes in gertty now. | 14:20 |
| anteaya | ah | 14:20 |
| *** mwagner_lap has joined #openstack-infra | 14:20 | |
| jeblair | anteaya: you might want to subscribe to a busier project | 14:20 |
| jhesketh_ | jeblair: with the new gerrit I'd like to get the zuul overlay stuff happening.. if you find yourself with some spare time (heh, so sorry!) it'd be awesome to get the zuul javascript stuff merged (tail is now here: https://review.openstack.org/#/c/91316/) | 14:20 |
| fungi | jeblair: agreed. perhaps git-review could have a configurable option to check whether the current patchset in gerrit is workflow -1 prior to uploading a new patchset and then automatically reapply it if so | 14:20 |
| jeblair | jhesketh_: yeah, though the hold-up wasn't gerrit -- it was zuul. | 14:21 |
| anteaya | jeblair: ah ha, it is the list page that eats up cpu | 14:21 |
| *** wchrisj has joined #openstack-infra | 14:21 | |
| *** wchrisj has left #openstack-infra | 14:21 | |
| jhesketh_ | oh I just meant now gerrit is done it'd be a nice feature to have | 14:21 |
| jhesketh_ | jeblair: by zuul, do you mean because you wanted to get the javascript stuff less duplicated | 14:22 |
| anteaya | I moved back to the list page, I had subscribed to openstack/tempest earlier and now the highest I can see on the list is python-marconiclient | 14:22 |
| jeblair | jhesketh_: yep -- i consider that a hard requirement before we really start on it -- because i don't want three javascript things that no one maintains. :) one is plenty. :) | 14:22 |
| anteaya | it finished giving me the full list and has calmed down again | 14:23 |
| anteaya | nope, navigating the list with my arrow keys, up and down arrow seems to have set it off again | 14:23 |
| jhesketh_ | jeblair: you'll have to show me how you trick people into doing javascript ;-P | 14:23 |
| jhesketh_ | (ie me) | 14:23 |
| jeblair | jhesketh_: i'm so good at it i tricked myself into it. | 14:24 |
| jhesketh_ | haha | 14:24 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Add queue-name parameter to job https://review.openstack.org/81320 | 14:25 |
| jeblair | jhesketh_: anyway, i'll try to get to it; i currently have 82 unreviewed changes in config though, so it might be a little bit. | 14:25 |
| jhesketh_ | jeblair: totally understand, I realise it's pretty low priority | 14:26 |
| jhesketh_ | and should be | 14:26 |
| anteaya | also after subscribing, navigation is frozen and cpu usage goes up to around 105% | 14:26 |
| jeblair | fungi: can you give 63190 the once over and aprv? | 14:26 |
| anteaya | perhaps that is to be expected? | 14:26 |
| jeblair | anteaya: not too surprising -- it's not the most efficient program, yet. | 14:27 |
| *** zns has quit IRC | 14:28 | |
| *** creiht has joined #openstack-infra | 14:29 | |
| creiht | did the ips of review.openstack.org change? | 14:29 |
| anteaya | jeblair: okay, continuing to poke around | 14:29 |
| jeblair | creiht: yes | 14:29 |
| anteaya | so far, I am very impressed | 14:29 |
| creiht | I'm getting ssh errors that the signature isn't the same | 14:29 |
| *** david-lyle has joined #openstack-infra | 14:29 | |
| jeblair | creiht: see https://wiki.openstack.org/wiki/GerritUpgrade | 14:29 |
| creiht | hehe | 14:29 |
| creiht | ok | 14:29 |
| creiht | thx | 14:30 |
| jeblair | np | 14:30 |
| derekh | fungi: I see the f20 nodes got built (at least the logs show no errors) thanks for helping it through, no we just have todo something useful with them | 14:30 |
| creiht | jeblair: since I'm here... :) | 14:30 |
| derekh | *now | 14:30 |
| creiht | jeblair: did you all and notmyname get the versioning stuff figured out for swift? | 14:30 |
| jhesketh_ | does it bother anybody else that the verified column is not leftmost in gerrit? | 14:30 |
| anteaya | yes | 14:30 |
| anteaya | right now we are stuck with it | 14:31 |
| *** nati_ueno has joined #openstack-infra | 14:31 | |
| jhesketh_ | it bothers me enough to suggest we rename it CI and go through the pain of fixing all 3rd party systems | 14:31 |
| anteaya | it is on the list of things we would like to address | 14:31 |
| anteaya | wow | 14:31 |
| jhesketh_ | anteaya: upstream with gerrit? | 14:31 |
| anteaya | that is quite the standard of measurement for your pain | 14:31 |
| jhesketh_ | well besides 3rd parties we can do the rest in one commit on config | 14:31 |
| anteaya | jhesketh_: not sure, probably give upstream a try | 14:32 |
| jhesketh_ | it's hurting my brain trying to scan what I need to review | 14:32 |
| anteaya | jhesketh_: /me nods | 14:32 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories https://review.openstack.org/70373 | 14:32 |
| *** gondoi is now known as zz_gondoi | 14:32 | |
| jhesketh_ | I fear, from what I've heard anyway, that if we went with upstream it'd take forever before it actually changed by which point I'd be used to the new order | 14:32 |
| * anteaya nods | 14:32 | |
| anteaya | very possible | 14:32 |
| * jhesketh_ nearly submitted a patch to rename the column this afternoon | 14:33 | |
| anteaya | jhesketh_: you might like to read this: http://git.openstack.org/cgit/stackforge/gertty/tree/README.rst | 14:33 |
| jhesketh_ | oh yeah, I've seen that :-) | 14:33 |
| *** zns has joined #openstack-infra | 14:33 | |
| anteaya | jeblair's newest invention | 14:33 |
| hashar | oh men | 14:33 |
| jhesketh_ | haven't tried it yet | 14:33 |
| anteaya | I'm playing with it | 14:33 |
| fungi | creiht: i do not believe it's been solved yet. i took a peek at the failure log for the tag merge job and for some reason it's ending up with more than just the merge commit staged (which is causing git-review to abort). i'll see if i can tell what's going on there | 14:33 |
| anteaya | very easy on teh eyes | 14:33 |
| creiht | fungi: ok cool | 14:33 |
| creiht | thanks for taking a look at it | 14:33 |
| creiht | was just curious where it was | 14:34 |
| anteaya | and since jeblair wrote it, I'm sure you can make it look however you want | 14:34 |
| fungi | derekh: yes, i was able to get the image to build with your patch applied, and then nodes were launched from it, so should be all set for some trial jobs | 14:34 |
| anteaya | jhesketh_: the listing and subscribing process is cpu heavy right now, and syncing takes a bit | 14:35 |
| anteaya | jhesketh_: but it might be more satisfying than what you are currently staring at | 14:35 |
| fungi | creiht: for the moment, this is the situation... http://logs.openstack.org/a8/a88690e47e5ec4a39603b42ca992c4f6129834da/release/swift-merge-release-tags/25429aa/console.html.gz | 14:36 |
| derekh | fungi: cool, you've seen the one I am proposing, anybody else want to take a look at https://review.openstack.org/#/c/86910/ ? | 14:36 |
| YorikSar | hashar, jeblair, fungi: I think that the best place for WIP-stickiness is in git-review now. We shouldn't stick all -1s in Gerrit because there is (and should be) more than one person to set WIP. Creatign a new plugin for Gerrit might be a good idea but it sounds like it won't come around shortly. | 14:36 |
| anteaya | so what does gertty consider an unreviewed change? any status:open patch on a repo? or do I have to have subscribed to a patch for it to be considered unreviewed for gertty? | 14:37 |
| jhesketh_ | okay, I'm off | 14:37 |
| jhesketh_ | night all! | 14:37 |
| anteaya | jhesketh_: night | 14:37 |
| anteaya | sleep well | 14:37 |
| jeblair | YorikSar: the plugin exists, it just requires 2.9 to be fully functional | 14:38 |
| jeblair | fungi: huh, i could have sworn i wrote a patch to fix the merge tag thing. 1 min. | 14:38 |
| *** zz_gondoi is now known as gondoi | 14:39 | |
| YorikSar | jeblair: Oh. Good :) But I think we won't get 2.9 upgrade until Juno... | 14:39 |
| jeblair | fungi: the script is already fixed. | 14:40 |
| *** wenlock has joined #openstack-infra | 14:40 | |
| fungi | jeblair: was it fixed after the 17th | 14:40 |
| fungi | ? | 14:40 |
| jeblair | fungi: https://review.openstack.org/#/c/88309/ | 14:41 |
| jeblair | fungi: yes | 14:41 |
| fungi | okay | 14:41 |
| jeblair | fungi: because of that failure | 14:41 |
| fungi | and just not retriggered yet i guess | 14:41 |
| jeblair | fungi: i think mordred was going to run the job again. | 14:41 |
| openstackgerrit | A change was merged to openstack-dev/pbr: Make source configurable when using --coverage https://review.openstack.org/89205 | 14:41 |
| fungi | i wonder now if it will encounter new problems since swift's milestone-proposed branch has been deleted in that time | 14:41 |
| fungi | i'll give the retrigger a shot real quicklike | 14:42 |
| *** dangers is now known as dangers_away | 14:43 | |
| openstackgerrit | A change was merged to openstack-infra/config: Create a swap partition if needed for new servers https://review.openstack.org/63190 | 14:43 |
| *** tteggel_ has quit IRC | 14:43 | |
| anteaya | jeblair: emailed you a screenshot of what gertty looks like for me | 14:43 |
| *** tteggel_ has joined #openstack-infra | 14:44 | |
| *** tteggel_ is now known as tteggel | 14:44 | |
| *** ildikov_ has quit IRC | 14:44 | |
| *** ildikov_ has joined #openstack-infra | 14:44 | |
| fungi | progress! http://logs.openstack.org/a8/a88690e47e5ec4a39603b42ca992c4f6129834da/release/swift-merge-release-tags/25429aa,1/console.html | 14:45 |
| fungi | though now we have an acl issue, i think, courtesy of gerrit upgrade | 14:45 |
| jeblair | yay! | 14:45 |
| fungi | and i need to run out to a quick appointment, but will have a look at that as soon as i return | 14:46 |
| anteaya | happy appointment | 14:46 |
| tnurlygayanov | hm | 14:46 |
| *** pcrews_ has joined #openstack-infra | 14:46 | |
| anteaya | oh I am out for an appointment myself this afternoon, leaving in about 3 hours | 14:46 |
| tnurlygayanov | after some fixes with setup.py / setup.cfg we can see that pbr want to build release 0.0.1.1 | 14:47 |
| tnurlygayanov | how we can say to pbr that we want release 0.0.2? | 14:47 |
| tnurlygayanov | we just run: python setup.py sdist - and after that we can see new tarball with version 0.0.1.1 | 14:48 |
| *** atiwari has joined #openstack-infra | 14:49 | |
| tnurlygayanov | we want to make a release of Mistral: https://github.com/stackforge/mistral | 14:49 |
| anteaya | tnurlygayanov: I know so little about pbr I am useless to you | 14:50 |
| anteaya | hopefully someone with more knowledge will show up soon | 14:50 |
| tnurlygayanov | anteaya: ok, thank you, I will wait :) | 14:51 |
| jeblair | tnurlygayanov: you just tag 0.0.2 and push that | 14:53 |
| jeblair | tnurlygayanov: pbr does tag-based post-versioning by default | 14:53 |
| *** _nadya_ has quit IRC | 14:54 | |
| openstackgerrit | A change was merged to openstack-infra/config: Create propose_translation_update for horizon https://review.openstack.org/82176 | 14:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Create propose_translation_update for horizon https://review.openstack.org/82176 | 14:55 |
| *** andreaf has quit IRC | 14:56 | |
| *** rcarrill` has joined #openstack-infra | 14:56 | |
| openstackgerrit | Antoine Musso proposed a change to stackforge/gertty: Print a friendly user message when conf is missing https://review.openstack.org/91385 | 14:56 |
| jeblair | that was weird | 14:56 |
| hashar | oh man, where is the tox.ini ! :-D | 14:57 |
| *** rcarrillocruz has quit IRC | 14:58 | |
| *** msabramo has joined #openstack-infra | 14:58 | |
| openstackgerrit | A change was merged to stackforge/gertty: Print a friendly user message when conf is missing https://review.openstack.org/91385 | 14:59 |
| jeblair | i think making it installable is probably a first order requirement for that. :) | 15:00 |
| anteaya | I'm going with gerritbot had a hiccup | 15:00 |
| jeblair | anteaya: glitch in the matrix | 15:00 |
| anteaya | apparently | 15:00 |
| anteaya | invoke gertty gerrit | 15:02 |
| anteaya | hashar: that gets it to run for you? | 15:02 |
| jeblair | fyi, i'm not really interested in adding pep8 to gertty for a while -- at least not until it settles down a bit. | 15:02 |
| anteaya | k | 15:03 |
| hashar | anteaya: yeah yeah playing with it right now :-] | 15:03 |
| hashar | jeblair: dont you have pep8 errors reported in your text editor ? | 15:03 |
| jeblair | (this is a fun project for me, and hey, it's on stackforge) | 15:03 |
| anteaya | hashar: cool | 15:04 |
| anteaya | it is great | 15:04 |
| jeblair | hashar: nope! i don't like computers bossing me around. ;) | 15:04 |
| *** Ajaeger has quit IRC | 15:04 | |
| *** jlibosva has quit IRC | 15:04 | |
| anteaya | right now crtl-C doesn't kill it, it stops gertty view but logging still happens and I don't get my prompt back | 15:04 |
| openstackgerrit | Cedric Brandily proposed a change to openstack-infra/git-review: Correct test_remote to support branchs without upstream https://review.openstack.org/83798 | 15:04 |
| anteaya | I have to track it down with ps aux and kill it | 15:05 |
| jeblair | anteaya: try ctrl-\ | 15:05 |
| anteaya | k | 15:05 |
| jeblair | harlowja_away: pointed out some threads that should be daemonized yesterday. also, it needs an actual quit command added. :) | 15:05 |
| *** zns has quit IRC | 15:06 | |
| anteaya | hashar: gertty gerrit doesn't work for me: http://paste.openstack.org/show/77997/ how do you get it working for yourself? | 15:06 |
| hashar | anteaya: how it does not work yet :] | 15:06 |
| *** zns has joined #openstack-infra | 15:06 | |
| anteaya | hashar: oh | 15:07 |
| anteaya | ctrl-\ works | 15:07 |
| jeblair | hashar: also, if you omit the name of the gerrit, it will just use the first one in the file | 15:07 |
| hashar | anteaya: you want: python gertty/gertty.py | 15:07 |
| anteaya | yes | 15:07 |
| anteaya | that is what I am using now, just wondered if I was missing something | 15:07 |
| *** esker has quit IRC | 15:08 | |
| anteaya | the docs are ahead of the code, that doesn't happen often | 15:08 |
| *** esker has joined #openstack-infra | 15:08 | |
| anteaya | hashar: what do you see in your screen when you select a view for a subscribed repo? | 15:09 |
| *** dkranz has quit IRC | 15:09 | |
| hashar | well | 15:11 |
| hashar | it currently tries to fetch /ra/projects/ instead of /r/projects (extra a) | 15:11 |
| *** jgallard has joined #openstack-infra | 15:11 | |
| anteaya | unsubscribing from a repo and then subscriping again to the same repo eats up cpu as well | 15:11 |
| anteaya | ah | 15:11 |
| *** mkoderer has quit IRC | 15:12 | |
| jeblair | hashar: i think the url in the config file should have a trailing slash | 15:12 |
| portante | can somebody offer guidance on how to get a new swift gate/check job in place? We recently added a new tox target, "tox -e func". | 15:13 |
| portante | i imagine it would run like the existing swift py26 and py27 unit tests | 15:13 |
| anteaya | portante: do you have a link to the patch? | 15:13 |
| portante | sec | 15:13 |
| *** ildikov_ has quit IRC | 15:13 | |
| hashar | jeblair: and there is an extra 'a/' inserted between the url and /projects/ http://paste.openstack.org/show/77998/ :D | 15:13 |
| portante | anteaya: https://review.openstack.org/66108 | 15:13 |
| jeblair | hashar: that's to make it authenticated | 15:14 |
| *** signed8bit has quit IRC | 15:14 | |
| chmouel | portante: i think it just need to be added to the envlist | 15:14 |
| chmouel | portante: http://pastie.org/pastes/9126738/text?key=3gyawvps2y9ycoyomvqsg | 15:14 |
| portante | so is that a swift tox.ini change? | 15:15 |
| jeblair | hashar: if the url ended with / it would be host.com/r/a/projects which should be correct for your setup i think | 15:15 |
| portante | I thought I already added func to the tox.ini change | 15:15 |
| *** signed8bit has joined #openstack-infra | 15:15 | |
| jeblair | no, we don't run the envlist -- that's just for convenience for developers on their local workstations (so you can say "tox" and run whatever the project admins think is appropriate) | 15:16 |
| hashar | jeblair: ah that is the API entry point. Thanks! | 15:16 |
| jeblair | portante: one sec and i'll grab some links | 15:17 |
| anteaya | hashar: so it needs to be r/a not ra/ ? | 15:17 |
| jeblair | portante: you'll want to add a job definition to this file: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/swift-functional.yaml | 15:17 |
| jeblair | portante: (more to come, gimme a sec) | 15:18 |
| portante | sec | 15:18 |
| jeblair | min | 15:18 |
| sdague | so here is a write up of some of the gerrit query changes and how to use them - https://dague.net/2014/04/30/helpful-gerrit-queries-gerrit-2-8-edition/ | 15:18 |
| sdague | figured that would be helpful for people to navigate the transition | 15:18 |
| openstackgerrit | Antoine Musso proposed a change to stackforge/gertty: Properly register password from prompt https://review.openstack.org/91392 | 15:18 |
| openstackgerrit | Antoine Musso proposed a change to stackforge/gertty: Normalize URL to ends with a slash https://review.openstack.org/91393 | 15:18 |
| *** pcrews_ has quit IRC | 15:19 | |
| *** thedodd has joined #openstack-infra | 15:19 | |
| *** doddstack has joined #openstack-infra | 15:20 | |
| jeblair | wow, we don't have any run-tox jobs right now... where did they go? | 15:20 |
| anteaya | hashar: the importance of 91392 is lost on me | 15:21 |
| anteaya | missing jobs? | 15:21 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Modularize to enable easier testing https://review.openstack.org/90572 | 15:21 |
| jeblair | oh they changed to run-unittests, right | 15:21 |
| hashar | anteaya: the 'noop' job is the default one. I guess James did not had the default python jobs in Jenkins and Zuul yet :] | 15:21 |
| *** WaltNik has quit IRC | 15:22 | |
| *** talluri has quit IRC | 15:22 | |
| jeblair | portante: strike my earlier suggestion | 15:22 |
| jeblair | portante: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/misc.yaml#n42 | 15:22 |
| *** che-arne has quit IRC | 15:22 | |
| jeblair | portante: i think that job defn (gate-{name}-unittests-{env}) may do what you need... | 15:23 |
| tnurlygayanov | so, problem with tags and tarballs versions was fixed (need just merge all fixes and push tags) | 15:23 |
| *** thedodd has quit IRC | 15:23 | |
| jeblair | portante: add a stanza like this: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml#n244 | 15:24 |
| jeblair | portante: to the swift project in that file, and set env = 'func' | 15:24 |
| *** dkranz has joined #openstack-infra | 15:25 | |
| *** signed8bit has quit IRC | 15:25 | |
| jeblair | portante: then add 'gate-swift-unittests-func' to the pipelines you want it in here: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n1022 | 15:25 |
| JayF | So I just figured out a big problem with the new gerrit | 15:26 |
| JayF | we're getting SPF softfails | 15:26 |
| JayF | which is why all the new mail is going into my spam folder | 15:26 |
| jeblair | portante: finally, you may want to add a job stanza with voting:false to this section: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n341 | 15:26 |
| portante | jeblair: where is the definition of the job that results here: http://logs.openstack.org/08/66108/25/gate/gate-swift-python27/1b03f8a/ | 15:26 |
| portante | ? | 15:26 |
| portante | because that is just doing "tox -epy27", and the new one would just be "tox -efunc" | 15:27 |
| jeblair | portante: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/python-jobs.yaml#n79 | 15:27 |
| JayF | openstack.org descriptive text "v=spf1 include:sendgrid.net ~all" <-- and it appears the new gerrit installation is sending locally, not from sendgrid | 15:27 |
| JayF | I'd strongly suggest dropping the SPF record or adding 23.253.232.87 to it | 15:27 |
| *** signed8bit has joined #openstack-infra | 15:28 | |
| jeblair | JayF: the same was true for the old server | 15:28 |
| JayF | Hmm. Then I still wonder why it's going to junk instead of inbox since the server change (on outlook.com hosted mail, so probably everything Microsoft, since Office 365/Hotmail share incoming mail infra afaict) | 15:29 |
| JayF | Either way, this seems like something that's nearly trivial to fix? Do we have automation against DNS where I could file a merge request, or is it all hand-configured? | 15:29 |
| *** eharney has quit IRC | 15:30 | |
| jeblair | JayF: it's possible they incorporate reputation into softfail, and the new server has little/no reputation now. | 15:30 |
| anteaya | hashar: so what is the purpose of https://review.openstack.org/#/c/91392/1 what is it fixing? | 15:30 |
| jeblair | JayF: unfortunately, no dns automation :( | 15:30 |
| JayF | So would an email to the list proposing a revised SPF record be the best route? | 15:30 |
| anteaya | hashar: it changes the scope of the input, okay sorry | 15:30 |
| *** mrodden has quit IRC | 15:31 | |
| hashar | anteaya: if you are missing the 'password' field, it asks for the password by prompting you. But your input is not registered in the config object :-( | 15:31 |
| jeblair | JayF: yeah | 15:31 |
| anteaya | hashar: okay, i see that now, sorry | 15:31 |
| anteaya | makes sense | 15:31 |
| JayF | jeblair: will-do. Any other mail sending servers in the infra world that I might not know about? | 15:31 |
| *** sdake_ has joined #openstack-infra | 15:32 | |
| *** sdake_ has quit IRC | 15:32 | |
| *** sdake_ has joined #openstack-infra | 15:32 | |
| jeblair | JayF: lists.o.o. also, they all send mail to me. | 15:32 |
| *** _nadya_ has joined #openstack-infra | 15:32 | |
| jeblair | i think openstackid-dev does, probably groups. | 15:32 |
| JayF | jeblair: for instance, I see MX for openstack.org goes to Rackspace Email & Apps (woo, I helped build this), does anyone send from those accounts? | 15:32 |
| jeblair | JayF: yep. | 15:32 |
| zaro | morning | 15:33 |
| anteaya | morning zaro | 15:33 |
| jeblair | JayF: honestly, it should probably be +all or ~all. | 15:33 |
| JayF | It's already ~all | 15:33 |
| JayF | but softfailing anything not from sendgrid seems worse than +all given that's all that's in the record | 15:33 |
| jeblair | JayF: yeah. i think the common interpretation of ~all is to accept but give scrutiny -- so i think the issue is less that we're saying "anything not sendgrid bad" and more that the new server is sending a lot of mail with no reputation yet. | 15:34 |
| JayF | I'll take a look at it though, I've worked with mail for a while and it won't take long to assemble a better SPF record. | 15:34 |
| JayF | jeblair: IME, working in email, a softfail is treated much more harshly than that, especially nowadays | 15:34 |
| jeblair | JayF: do you git openstack-dev@lists.o.o email? | 15:35 |
| JayF | although reputation does matter quite a bit, if you pass the SPF record, you should inherit the reputation of the domain, which should be pristine | 15:35 |
| JayF | I get that email, and it does go to my inbox properly | 15:35 |
| portante | jeblair, chmouel: so is this then just as easy as modifying swift's tox.ini file to have "func" in the envlist and we're done? | 15:35 |
| JayF | I'm betting it doesn't softfail because lists.o.o doesn't have SPF | 15:35 |
| portante | cause that would be cool | 15:35 |
| portante | and simple | 15:35 |
| JayF | Authentication-Results: hotmail.com; spf=none (sender IP is 50.56.173.222) | 15:35 |
| JayF | vs spf=softfail for gerrit | 15:35 |
| jeblair | portante: no, i explained earlier why that's not the case. see scrollback. i also detailed what you should do to add the job. | 15:36 |
| portante | k, sorry I did not catch that | 15:36 |
| jeblair | JayF: oh, this is because of <review@openstack.org> ? | 15:37 |
| anteaya | portante: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-30.log timestamp 2014-04-30T15:22:57 | 15:37 |
| JayF | yes, absolutely | 15:37 |
| JayF | so it's hitting the openstack.org SPF record, because that's the domain in the email | 15:37 |
| JayF | if it were, for instance, review@review.openstack.org, spf wouldn't be evaluated and it would pass | 15:38 |
| JayF | well, not pass, but much better than softfail | 15:38 |
| portante | anteaya: thanks | 15:38 |
| anteaya | np | 15:39 |
| JayF | something like "v=spf1 a:review.openstack.org include:sendgrid.net include:emailsrvr.com ~all" would be better and prevent anyone from having to worry about it if gerrit servers change again | 15:39 |
| JayF | I'll look at openstackid-dev as well if I've ever gotten any email from it and pop a mail out to the list | 15:39 |
| *** msabramo has quit IRC | 15:40 | |
| *** gyee has joined #openstack-infra | 15:40 | |
| *** vponomaryov has left #openstack-infra | 15:40 | |
| jeblair | JayF: or we could do +all instead of whack-a-mole. :) | 15:42 |
| *** che-arne has joined #openstack-infra | 15:42 | |
| JayF | I would strongly prefer we get a good SPF record in place with the flexibility to keep things working. If you add +all, then you might as well remove the SPF record altogether :( | 15:43 |
| *** msabramo has joined #openstack-infra | 15:43 | |
| JayF | The A++ solution would be to get all outgoing mail going through Mailgun or Sendgrid, but that's a much larger undertaking than just trying to improve the SPF record :). Besides, a reasonably setup SPF record with a:some.host.name type entries should only have to be modified when new services that send mail come online on the openstack.org domain (and not a subdomain, like lists.o.o) | 15:44 |
| jeblair | JayF: i don't agree at all that is a good solution. i'm strongly opposed to paid mail providers being email gatekeepers. | 15:45 |
| JayF | I understand that philosophy, and that's why my actual proposal is to fix the SPF record :). | 15:45 |
| JayF | Easier to trust Mailgun when I can go give their whole tech staff high fives a few steps away. | 15:45 |
| fungi | mmmscrollback | 15:49 |
| *** mrodden has joined #openstack-infra | 15:49 | |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Make ordereddict optional https://review.openstack.org/91401 | 15:50 |
| hashar | jeblair: can you exit gertty by pressing Ctrl + C ? The app seems to terminate but I am left with a leftover Python process (might be my setup) | 15:52 |
| *** serverascode has quit IRC | 15:52 | |
| *** UtahDave has joined #openstack-infra | 15:52 | |
| mugsie | SergeyLukjanov: i fixed that ACL issue in https://review.openstack.org/#/c/87540/ - is there anything else you think I need to do? | 15:52 |
| jeblair | hashar: no, it needs some threads marked daemon, and also an actual exit command. i'm using ctrl-\ for now | 15:52 |
| anteaya | hashar: ctrl \ | 15:53 |
| hashar | oh | 15:53 |
| openstackgerrit | A change was merged to stackforge/gertty: Make ordereddict optional https://review.openstack.org/91401 | 15:53 |
| anteaya | I asked the same above | 15:53 |
| hashar | I can't do that magic escape commands with my french keyboard :-/ | 15:53 |
| anteaya | :( | 15:53 |
| hashar | that gives me : | 15:53 |
| *** talluri has joined #openstack-infra | 15:53 | |
| hashar | to get backslashes I have to alt + shift + : , doing ctrl + alt + shift + : just yield : | 15:54 |
| *** ihrachyshka has quit IRC | 15:54 | |
| hashar | I can surely redefine it | 15:54 |
| *** serverascode has joined #openstack-infra | 15:54 | |
| anteaya | jeblair: I pulled in that last change | 15:54 |
| anteaya | now I can't get to the list page | 15:55 |
| anteaya | pressing l gets me no change | 15:55 |
| anteaya | and somehow I got subscribed to openstack/sahara-dashboard because now I am looking at the repo view | 15:56 |
| anteaya | and now I have the list view | 15:56 |
| anteaya | maybe there is lag? | 15:57 |
| *** talluri has quit IRC | 15:58 | |
| anteaya | ctrl + alt + shift + : wow, do you have to get help from the cat for that one? | 15:59 |
| *** andreykurilin_ has joined #openstack-infra | 15:59 | |
| fungi | JayF: jeblair: the downside to having gerrit send as something@review.o.o is that unless we also add an mx for review.o.o to somewhere which will accept bounces for that address, sender verification callouts on recipient mtas will start failing (different set of span-prevention-tripwires to deal with) | 15:59 |
| *** KurtMartin is now known as kmartin | 15:59 | |
| fungi | s/span/spam/ | 16:00 |
| JayF | fungi: that's a good point. I just think we should update the SPF record and everything will be fine :) | 16:00 |
| hashar | anteaya: well I should get a qwerty keyboard. Or figure out how to associate a different keystroke to whatever ctrl + \ does :] | 16:00 |
| anteaya | or train the cat | 16:00 |
| anteaya | the cat invokation command | 16:01 |
| anteaya | invocation | 16:01 |
| *** packet has joined #openstack-infra | 16:01 | |
| hashar | I managed to get something! | 16:01 |
| hashar | but have to drop /a/ from the URL :/ | 16:01 |
| anteaya | interesting | 16:02 |
| hashar | I can't connect to https://gerrit.wikimedia.org/r/a/ (the user/pass is rejected hehe) | 16:02 |
| hashar | but https://gerrit.wikimedia.org/r/projects/ works fine | 16:02 |
| *** packet has quit IRC | 16:03 | |
| anteaya | huh | 16:03 |
| anteaya | what file did you make that change in? | 16:03 |
| *** chandan_kumar has quit IRC | 16:03 | |
| *** sdake_ has quit IRC | 16:03 | |
| jeblair | hashar: unauthenticated access isn't going to work for most of what gertty does; you'll want to fix that. | 16:03 |
| *** dkranz has quit IRC | 16:04 | |
| hashar | jeblair: I think I got authenticated at that address | 16:04 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:04 | |
| *** achuprin has quit IRC | 16:05 | |
| *** eharney has joined #openstack-infra | 16:05 | |
| *** afazekas has quit IRC | 16:05 | |
| *** packet has joined #openstack-infra | 16:05 | |
| *** packet has quit IRC | 16:05 | |
| *** packet has joined #openstack-infra | 16:06 | |
| *** sdake_ has joined #openstack-infra | 16:07 | |
| *** marcoemorais has joined #openstack-infra | 16:08 | |
| *** zehicle_at_dell has quit IRC | 16:08 | |
| jeblair | hashar: https://gerrit.wikimedia.org/r/Documentation/rest-api.html#authentication | 16:09 |
| hashar | jeblair: ah thanks. was looking for that :] | 16:10 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/requirements: Add oslo.db library https://review.openstack.org/91407 | 16:10 |
| *** viktors has quit IRC | 16:12 | |
| *** signed8bit has quit IRC | 16:13 | |
| *** signed8bit has joined #openstack-infra | 16:14 | |
| fungi | jeblair: mordred: so the current acl situation with the tag-merge jobs is that the proposal bot account is not in a group which can push merge commits... presently only project bootstrappers has that... though looking more closely i think the release managers are also going to have a hard time pushing tags as well (right now they're allowed annotated tags but not signed tags) | 16:14 |
| fungi | to deal with the tag-merge though, i suggest we add a specific group we can put the proposal bot account in for that access (unless you think one of the existing groups is well suited, though they don't seem like it to me) | 16:16 |
| *** nati_ueno has quit IRC | 16:16 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-dev/pbr: pbr/testr_command.py: Add logging https://review.openstack.org/89203 | 16:16 |
| *** dkranz has joined #openstack-infra | 16:17 | |
| *** weshay has quit IRC | 16:18 | |
| *** rcarrillocruz has joined #openstack-infra | 16:18 | |
| fungi | but separately, i'll switch the annotated tag permissions to signed tag in all-projects unless there are any objections | 16:18 |
| jeblair | fungi: agreed on all counts (i wonder how this was supposed to work before..) | 16:19 |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Fix editing inline comments https://review.openstack.org/91410 | 16:20 |
| fungi | i have a suspicion that the merge perms were previously being granted by some other permission which got stricter between 2.4 and 2.8 | 16:20 |
| *** talluri has joined #openstack-infra | 16:20 | |
| *** rcarrill` has quit IRC | 16:20 | |
| fungi | for example, the proposal bot has batch priority in global caps. not sure what that means exactly... | 16:21 |
| *** hogepodge has joined #openstack-infra | 16:21 | |
| *** marun has quit IRC | 16:23 | |
| kiall | jeblair: Heya - Hows the tag situation? Should I bother you again tomorrow? ;) | 16:23 |
| *** weshay has joined #openstack-infra | 16:24 | |
| *** jpich has quit IRC | 16:24 | |
| *** etoews has joined #openstack-infra | 16:24 | |
| *** hemna__ is now known as hemna | 16:24 | |
| *** talluri has quit IRC | 16:25 | |
| *** bhuvan has quit IRC | 16:25 | |
| *** manishg has joined #openstack-infra | 16:27 | |
| *** sdake_ has quit IRC | 16:29 | |
| jeblair | kiall: i think we've got it worked out. do you have a link to the failed job? | 16:29 |
| kiall | Yep - It was https://jenkins01.openstack.org/job/designate-tarball/3/ | 16:30 |
| kiall | But the tag is unsigned | 16:30 |
| kiall | so needs to be `rm -rf`'d | 16:30 |
| *** rakhmerov has quit IRC | 16:31 | |
| *** pcrews_ has joined #openstack-infra | 16:32 | |
| jeblair | kiall: here are our options: (a) just run the job again on the unsigned tag and publish that; (b) delete the unsigned tag so you can push a signed version; (c) ignore the unsigned tag and push a signed 2014.1.1 | 16:33 |
| anteaya | jeblair: is there any http response code handling in gertty? | 16:33 |
| *** markmc has quit IRC | 16:33 | |
| kiall | jeblair: (B) is ideal.. But sounds like the most painful for you? | 16:34 |
| jeblair | kiall: nah, it's okay. will be just a minute. | 16:34 |
| jeblair | anteaya: it might log something? | 16:34 |
| *** krtaylor has quit IRC | 16:35 | |
| jeblair | kiall: done | 16:36 |
| kiall | Cool | 16:36 |
| anteaya | jeblair: looking, I am probably not looking in the right place yet | 16:36 |
| *** hashar has quit IRC | 16:38 | |
| anteaya | I'm getting a lot of this in the logs: http://paste.openstack.org/show/78024/ is that due to the auth error, hashar? | 16:40 |
| *** hogepodge has quit IRC | 16:40 | |
| anteaya | oh he left | 16:40 |
| fungi | jeblair: mordred: notmyname: creiht: https://review.openstack.org/91417 | 16:40 |
| kiall | jeblair: Huumm - Pushed a new signed tag, missed the zuul status page link to the job, but no tarball produced | 16:41 |
| notmyname | fungi: thanks | 16:41 |
| *** crc32 has joined #openstack-infra | 16:41 | |
| *** dkliban is now known as dkliban_lunch | 16:41 | |
| kiall | jeblair: looks to be still a race somewhere? https://jenkins03.openstack.org/job/designate-tarball/2/console | 16:42 |
| fungi | notmyname: you're welcome. jeblair apparently fixed the job a while back, but it didn't get re-run and then some changes in acl behavior in new gerrit broke it again, but all working now i think | 16:42 |
| kiall | That's a signed tag, so re-triggering should be enough to kick it into action | 16:42 |
| *** chandan_kumar has joined #openstack-infra | 16:43 | |
| fungi | kiall: i strongly suspect the degree of staleness for hpcloud-az2 nodepool images is such that the fixed slave scripts are not present on that node | 16:43 |
| *** romcheg has joined #openstack-infra | 16:43 | |
| * fungi checks | 16:43 | |
| kiall | hah :) | 16:44 |
| *** romcheg has left #openstack-infra | 16:44 | |
| fungi | kiall: yeah, our most recent bare-precise image in hpcloud-az2 is 348.79 hours old | 16:44 |
| *** dizquierdo has quit IRC | 16:45 | |
| kiall | Nice - So failing to rebuild.. Annoying! | 16:45 |
| fungi | courtesy of the nova metadata server not being responsive, causing daily image updates to fail there most of the time | 16:45 |
| kiall | The "new regions" metadata service is much more reliable, if that's of any value in pointing out ;) | 16:46 |
| fungi | apparently puppet facter wants to query it, then gives up after not getting a response, then puppet treats that as a hard failure and nodepool has to abort the image update | 16:46 |
| kiall | (I know, pros and cons...!) | 16:46 |
| fungi | kiall: i can re-trigger the job and see if it gets lucky and runs on a less stale node | 16:47 |
| *** jp_at_hp has quit IRC | 16:48 | |
| anteaya | jeblair: yes the http response codes are in the log file: http://paste.openstack.org/show/78028/ | 16:48 |
| kiall | fungi: ah - roulette, let's play ;) | 16:48 |
| *** derekh has quit IRC | 16:48 | |
| *** matty_dubs|lunch is now known as matty_dubs | 16:48 | |
| jeblair | fungi: this is an important problem, maybe we should remove the stale images | 16:49 |
| *** bhuvan has joined #openstack-infra | 16:49 | |
| fungi | jeblair: i'm happy to do that. how old is too old? just remove bare-precise images? | 16:50 |
| fungi | if it's specifically the tag fixes we're concerned about, then bare-precise images prior to the script patch should cover it | 16:50 |
| fungi | kiall: got lucky that time... http://logs.openstack.org/e8/e888486ab133bef1b2020ad26e2bcba5df2944f7/release/designate-tarball/56af9e1,1/console.html | 16:51 |
| *** mbacchi has quit IRC | 16:51 | |
| *** hogepodge has joined #openstack-infra | 16:51 | |
| fungi | looks like the fix merged 04-29 20:30 | 16:52 |
| fungi | so any bare-precise images prior to today's scheduled image update should be deleted | 16:53 |
| *** annegentle has joined #openstack-infra | 16:53 | |
| jeblair | fungi: yeah | 16:53 |
| nibalizer | sorry i've been so afk yall | 16:53 |
| nibalizer | $REALJOB has moved into serious overdrive, doesn't leave much energy for OSS | 16:53 |
| *** harlowja_away is now known as harlowja | 16:54 | |
| fungi | nibalizer: sorry to hear that. on the other hand, the shackles of job security are often padded for comfort ;) | 16:55 |
| *** etoews has quit IRC | 16:57 | |
| *** yidclare has joined #openstack-infra | 16:57 | |
| kiall | fungi: I see a tarball :) Thanks! | 16:57 |
| kiall | and jeblair - thanks :) | 16:57 |
| anteaya | it looks like there needs to be username:password inserted in here somewhere for the url to be authorized: http://git.openstack.org/cgit/stackforge/gertty/tree/gertty/sync.py#n135 | 16:58 |
| *** praneshp has joined #openstack-infra | 16:59 | |
| fungi | okay, all bare-precise nodepool images older than yesterday's ggp patch are now gone, and any nodes they begat should die off to attrition fairly rapidly as well | 16:59 |
| *** mbacchi has joined #openstack-infra | 17:00 | |
| *** nati_ueno has joined #openstack-infra | 17:00 | |
| *** talluri has joined #openstack-infra | 17:00 | |
| *** jamielennox is now known as jamielennox|away | 17:00 | |
| jeblair | fungi: thanks; hopefully that will let the job running robots do their jobs (so we don't have to step in for them). | 17:00 |
| cody-somerville | How can I delete a bunch of changes from a pipeline? | 17:00 |
| fungi | jeblair: agreed | 17:01 |
| *** nati_ueno has quit IRC | 17:01 | |
| jeblair | cody-somerville: 3 options: (a) push new patchsets for all of them. (b) stop zuul. (c) write the 'dequeue' command line client | 17:01 |
| *** praneshp_ has joined #openstack-infra | 17:02 | |
| jeblair | cody-somerville: typically (a) is what we do around here; sometimes (b). | 17:02 |
| *** _nadya_ has quit IRC | 17:02 | |
| *** ominakov has quit IRC | 17:02 | |
| fungi | there are definitely times where (c) would have been great to have, since (a) and (b) are both pretty labor-intensive | 17:03 |
| jeblair | cody-somerville: (if they are git-dependent, you might only have to push a new patch for the first one) | 17:03 |
| *** nati_ueno has joined #openstack-infra | 17:03 | |
| *** hashar has joined #openstack-infra | 17:04 | |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Handle "No newline at end of file" and add --no-sync option https://review.openstack.org/91418 | 17:04 |
| *** praneshp has quit IRC | 17:04 | |
| *** praneshp_ is now known as praneshp | 17:04 | |
| *** hashar has quit IRC | 17:04 | |
| *** Ajaeger has joined #openstack-infra | 17:04 | |
| openstackgerrit | A change was merged to openstack-infra/config: Finish puppeting accessbot https://review.openstack.org/86440 | 17:05 |
| cody-somerville | So I have a bunch of jobs (~300) in the post pipeline for a bunch of deleted branches on the same repository. I also have a small number of changes unrelated in the check queue. The former seems to be making things get processed very slowly. I have a bunch of ready slaves but zuul is only slowly queuing up the -branch-tarball jobs (and sometimes -coverage jobs) for the deleted branches. It was speculated this might be be du | 17:06 |
| cody-somerville | e to the -branch-tarball job being configured not to run concurrently. | 17:06 |
| *** chandan_kumar has quit IRC | 17:06 | |
| *** _david_ has joined #openstack-infra | 17:06 | |
| *** SumitNaiksatam has quit IRC | 17:06 | |
| fungi | cody-somerville: do you have more than one jenkins master? if so, non-concurrency isn't guaranteed anyway | 17:07 |
| cody-somerville | I had about 20 -branch-tarball jobs queued up in Jenkins - and naturally it made sense they wouldn't be executing in parallel since specifically being configured not to. I thought maybe this was causing zuul to think all the slaves were busy and not dispatch more things that *could* run. | 17:08 |
| fungi | unless you only have one slave for the job with a single executor on it, in which case you're de-facto non-concurrent without actually configuring that option | 17:08 |
| cody-somerville | So I deleted the 20 or so -branch-tarball jobs from the jenkins queue but zuul continues to just slowly add -branch-tarball jobs again to jenkin's queue. | 17:08 |
| jeblair | cody-somerville: because we can't guarantee concurrency, we don't have any jobs configured in that way. you're in untested territory. however, i would guess the following things are happening: | 17:09 |
| *** jgallard has quit IRC | 17:10 | |
| *** Ryan_Lane has joined #openstack-infra | 17:10 | |
| jeblair | cody-somerville: a) jenkins gearman plugin doesn't know about the non-concurrency aspect and is scheduling jobs as fast as it can on specific nodes. this means those nodes are unavailable to run other jobs while jenkins waits to run a job on them. | 17:10 |
| *** annegentle has quit IRC | 17:10 | |
| jeblair | cody-somerville: that explains why your check jobs are not running while you appear to have nodes available (they aren't available, they are actually "running" post jobs, they just haven't actually started yet) | 17:10 |
| *** dangers_away is now known as dangers | 17:11 | |
| jeblair | cody-somerville: b) when you remove jobs from the jenkins queue, it would not surprise me if zuul interprets that as a transient jenkins error and reschedules them | 17:11 |
| harlowja | jeblair do u want flake8 and stuff running on gertty? | 17:11 |
| harlowja | just wondering :) | 17:11 |
| jeblair | harlowja: no, not for a while. when development slows down a bit. | 17:12 |
| dhellmann | did viktors ask to have the oslo-db-core team set up in gerrit? could someone add me to that group so I can add everyone else? | 17:12 |
| msabramo | what is gertty? | 17:12 |
| msabramo | gerrit on your tty? | 17:13 |
| jeblair | dhellmann: will do | 17:13 |
| jeblair | msabramo: you win! | 17:13 |
| dhellmann | jeblair: thanks! | 17:13 |
| harlowja | jeblair sounds good | 17:13 |
| msabramo | woo hoo! | 17:13 |
| msabramo | sounds like we all win | 17:13 |
| harlowja | msabramo its the next best thing after https://github.com/harlowja/gerrit_view/ :-P | 17:13 |
| jeblair | msabramo: http://git.openstack.org/cgit/stackforge/gertty/tree/README.rst | 17:13 |
| harlowja | that jeblair is coming up, 21st century technology | 17:13 |
| cody-somerville | ah | 17:13 |
| dhellmann | jeblair: I don't remember if there's an oslo-db-milestone or not, but I should be in that one, too :-) | 17:14 |
| harlowja | with lots of new agile hotness, lol | 17:14 |
| cody-somerville | the concurrency restriction on jobs was only removed recently | 17:14 |
| cody-somerville | https://github.com/openstack-infra/config/commit/ef40ea5b5d372ae30d0087727ce1fbf2d1c9f14e | 17:14 |
| fungi | msabramo: so much gerrit it all up in yuo tty | 17:14 |
| fungi | !!!1!eleven | 17:14 |
| openstack | fungi: Error: "!!1!eleven" is not a valid command. | 17:14 |
| fungi | openstack: you're so disagreeable sometimes | 17:15 |
| *** manishg has quit IRC | 17:15 | |
| jeblair | dhellmann: https://review.openstack.org/#/admin/projects/openstack/oslo.db,access no oslo-db-milestone, i think we missed that in review | 17:15 |
| jeblair | dhellmann: i added oslo-core to oslo-db-core -- see if that's enough for you to self-manage now | 17:15 |
| jeblair | dhellmann: https://review.openstack.org/#/admin/groups/331,members | 17:15 |
| jeblair | dhellmann: if not, i'll add you explicitly | 17:16 |
| dhellmann | jeblair: it's working, thanks! | 17:16 |
| sdague | jeblair / mordred : thoughts on putting the new dashboard in today? | 17:17 |
| dhellmann | jeblair: as far as the milestone group, I'll submit a change to make that oslo-core | 17:17 |
| dhellmann | or oslo-db-milestone, I guess | 17:17 |
| *** Alexei_987 has joined #openstack-infra | 17:18 | |
| jeblair | sdague: can you change 'main' to 'review-inbox'? | 17:19 |
| *** esker has quit IRC | 17:19 | |
| *** manishg has joined #openstack-infra | 17:19 | |
| sdague | sure, on review-dev? | 17:19 |
| *** SumitNaiksatam has joined #openstack-infra | 17:19 | |
| _david_ | gerrit Workflow label confiuration: how can i propose a change for it to openstack-infra-config? | 17:19 |
| jeblair | sdague: yeah | 17:19 |
| *** krtaylor has joined #openstack-infra | 17:19 | |
| sdague | jeblair: done | 17:20 |
| jeblair | _david_: i think it's still manually managed (mordred wants to fix that); what's the change? | 17:21 |
| jeblair | https://review.openstack.org/gitweb?p=All-Projects.git;a=blob;f=project.config;h=f067a64a8bcb2317c74144bf86694a2c9a3696e6;hb=c2f7b042f2bc2f6884930790b47b1d5c2c9abf0f | 17:21 |
| _david_ | jeblair, thanks, i see, i think we should set: copyMinScore = true for WIP state to enforce stickeness on upload new patch sets | 17:22 |
| _david_ | jeblair, Congrats on upgrade, well done ;-) | 17:22 |
| jeblair | _david_: thanks :) | 17:23 |
| openstackgerrit | Anita Kuno proposed a change to stackforge/gertty: Adds --digest flag to url required for http auth https://review.openstack.org/91421 | 17:23 |
| jeblair | _david_: we can't make it sticky right now -- because the core group can -1wip someone else's change, that means they can't remove that vote, only a new patchset clears that (or they would have to pester core reviewers, and that defeats the purpose of wip) | 17:24 |
| *** grantbow has joined #openstack-infra | 17:24 | |
| jeblair | _david_: i think instead we should make it easier for devs to express whether they want a new patchset to be wip when uploaded | 17:24 |
| jeblair | _david_: which probably means changing git-review to support it | 17:24 |
| jeblair | _david_: and, i'm hoping, if we move to the wip plugin in 2.9, maybe it could support something like /refs/wip/foo so that becomes a one-step process on upload | 17:25 |
| fungi | and adding wip control (back) to dit-review has been on the wishlist for a while anyway | 17:25 |
| fungi | s/dit/git/ | 17:25 |
| jeblair | fungi: i think you mean derp-review | 17:25 |
| *** andreykurilin_ has quit IRC | 17:25 | |
| fungi | i SO do | 17:25 |
| anteaya | jeblair: this change is possibly in the wrong place but that is the notion anyway: https://review.openstack.org/#/c/91421/ | 17:27 |
| anteaya | I have to get ready to head out for an appointment, gone for several hours | 17:28 |
| _david_ | jeblair, i thought that change owner can always remove wip state? | 17:28 |
| jeblair | _david_: i don't think they can remove someone else's vote though | 17:29 |
| Shrews | fungi: has anyone reported the LP bug hook doesn't appear to be working? | 17:29 |
| *** nati_ueno has quit IRC | 17:30 | |
| *** nati_ueno has joined #openstack-infra | 17:30 | |
| fungi | Shrews: you just reported it ;) | 17:30 |
| fungi | (in other words, first i've heard) | 17:30 |
| jeblair | _david_: that's not desirable -- it's a drawback of the review label approach; but it's mitigated by not having copyminscore | 17:30 |
| fungi | i'll have a look | 17:30 |
| Shrews | first! \o/ | 17:30 |
| fungi | nb4bug | 17:31 |
| *** amcrn has quit IRC | 17:31 | |
| *** nati_ueno has quit IRC | 17:31 | |
| *** dkliban_lunch is now known as dkliban | 17:31 | |
| *** nati_ueno has joined #openstack-infra | 17:31 | |
| Shrews | fungi: if you need an example: https://review.openstack.org/#/c/91366/ | 17:31 |
| fungi | Shrews: that helps, thanks | 17:31 |
| _david_ | jeblair, to be fair wip plugin would clear the state with new patch set upload too | 17:32 |
| fungi | Shrews: i'm not terribly surprised though... the gerrit hooks interface is relatively unstable and right now some of our hooks probably break when passed new options they don't recognize | 17:33 |
| jeblair | _david_: yeah, and i personally prefer the clear on new patchset, but at least then it becomes a discussion about what we should do vs what we must do. :) | 17:33 |
| jeblair | _david_: that's why i think the best place to fix this is at patch upload -- so the developer can decide | 17:33 |
| jeblair | i have to run out for a few mins | 17:34 |
| fungi | k | 17:34 |
| _david_ | jeblair, agreed, so let me check the options we have or can introduce upstream ... | 17:34 |
| jeblair | _david_: and thank you very much for your help with the upgrade. i think some of the patches we merged recently for it were the ones you uploaded earlier in the year. :) | 17:35 |
| fungi | _david_: yes, it was a HUGE help. thanks! | 17:36 |
| _david_ | jeblair, thank you guys back for yor help LibreOffice community two years ago ;-) | 17:36 |
| _david_ | fungi, thanks | 17:36 |
| *** _david_ has quit IRC | 17:36 | |
| *** dripton has quit IRC | 17:37 | |
| *** med_ has joined #openstack-infra | 17:43 | |
| *** david-lyle has quit IRC | 17:45 | |
| *** rcarrillocruz has quit IRC | 17:45 | |
| *** rcarrill` has joined #openstack-infra | 17:46 | |
| zaro | jeblair: would it not make sense to give core reviewers permission to unabandone any change? | 17:46 |
| fungi | zaro: i believe that was on our to do list now that it can be controlled with the acl | 17:47 |
| *** manishg has quit IRC | 17:47 | |
| zaro | fungi: if that permission is set then i think it would make sense to set copyMinScore ? | 17:48 |
| zaro | fungi: it seems appropriate to me to have to contact a core to unabadone a change that it's owned by yourself. | 17:48 |
| fungi | zaro: i'm not sure what copyminscore has to do with abandoned changes | 17:49 |
| fungi | zaro: the idea with abandonment was to allow core reviewers and change owners to abandon/unabandon changes | 17:49 |
| zaro | fungi: ohh i meant wip all along. | 17:49 |
| zaro | fungi: so un-wip not un-abandone | 17:50 |
| *** dripton has joined #openstack-infra | 17:50 | |
| fungi | wip is different with the workflow vote. if i as a core reviewer set your patchset to wip, you as the change owner have no ability to switch it back out of wip because you can't change my vote | 17:50 |
| *** melwitt has joined #openstack-infra | 17:50 | |
| zaro | fungi: ohh of course. | 17:51 |
| *** david-lyle has joined #openstack-infra | 17:51 | |
| openstackgerrit | Peter Portante proposed a change to openstack-infra/config: Add swift "in-process" functional test runs https://review.openstack.org/91424 | 17:52 |
| portante | chmouel: ^^^ | 17:52 |
| *** darraghb has quit IRC | 17:53 | |
| zaro | jeblair: i'm trying to setup gear on a VM to work with SSL certs, but I can't seem to get it to work. I mean i can't get a connection. would you be able to help me with that? | 17:53 |
| *** thomasbiege has joined #openstack-infra | 17:53 | |
| *** amcrn has joined #openstack-infra | 17:53 | |
| *** thomasbiege has quit IRC | 17:56 | |
| *** che-arne has quit IRC | 17:56 | |
| *** doddstack has quit IRC | 18:00 | |
| *** dkliban is now known as dkliban_afk | 18:02 | |
| *** zns has quit IRC | 18:03 | |
| *** yfried has joined #openstack-infra | 18:03 | |
| yfried | asking again - is there anyone here familiar with logstash? | 18:04 |
| yfried | jesusaurus: ? | 18:04 |
| yfried | I'm trying to set up logstash as a deamon | 18:04 |
| yfried | how is done on the gate? | 18:04 |
| *** marun has joined #openstack-infra | 18:05 | |
| *** jooools has quit IRC | 18:06 | |
| *** msabramo has left #openstack-infra | 18:06 | |
| fungi | yfried: we have a special agent which gets zmq notification from jenkins job completion and then retrieves and injects all the log files from that job | 18:07 |
| fungi | so it's a batch processing loader | 18:07 |
| *** nati_ueno has quit IRC | 18:07 | |
| fungi | it's not set up like a typical logserver | 18:07 |
| *** nati_ueno has joined #openstack-infra | 18:07 | |
| *** e0ne has joined #openstack-infra | 18:08 | |
| *** thomasbiege has joined #openstack-infra | 18:08 | |
| yfried | fungi: ok, different question: this is how logfiles are parsed https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb | 18:09 |
| yfried | fungi: but how do you tag logs according to job and build? | 18:09 |
| fungi | yfried: metadata provided in the 0mq messages coming from jenkins is interpreted by the loader | 18:10 |
| yfried | fungi: how? where? | 18:11 |
| fungi | yfried: http://git.openstack.org/cgit/openstack-infra/zmq-event-publisher/tree/README | 18:11 |
| *** ildikov has joined #openstack-infra | 18:12 | |
| yfried | fungi: tnx. I'll look into it | 18:13 |
| *** timrc is now known as timrc-afk | 18:14 | |
| yfried | fungi: another question - are the logs being parsed during the run, or are the sent at the end of a job? | 18:14 |
| *** morganfainberg_Z is now known as morganfainberg | 18:14 | |
| fungi | once the job completes, jenkins pulls logs off the slave and publishes them to a webserver for archival, then sends a zeromq message to the logstash processor notifying of their availability. the logstash processor downloads those files from the webserver and then starts feeding them into logstash for indexing | 18:15 |
| yfried | fungi: where is all of this documented? because it doesn't sound like what I've read in the short wiki for logstash.openstack | 18:17 |
| fungi | yfried: http://ci.openstack.org/logstash.html | 18:17 |
| yfried | fungi: yeah, that's the one I've read | 18:17 |
| yfried | doesn't seem like exactly what you described. | 18:19 |
| *** e0ne has quit IRC | 18:19 | |
| fungi | yfried: that page documents how logstash is integrated, but doesn't necessarily go into details on things happening on the jenkins side (the log archiving we're doing from that end predates any of the logstash implementation and is mostly unchanged \) | 18:19 |
| *** zns has joined #openstack-infra | 18:19 | |
| yfried | fungi: I also was unable to find the log-pusher mentioned there | 18:19 |
| fungi | yfried: the links to them aren't working? | 18:20 |
| *** zns_ has joined #openstack-infra | 18:20 | |
| fungi | aha | 18:20 |
| fungi | the urls there are lacking /cgit/ in them | 18:20 |
| *** nati_ueno has quit IRC | 18:20 | |
| *** nati_ueno has joined #openstack-infra | 18:21 | |
| fungi | no, nevermind. perhaps they were moved and the docs not updated | 18:21 |
| *** thomasbiege has quit IRC | 18:21 | |
| *** ominakov has joined #openstack-infra | 18:21 | |
| *** zns has quit IRC | 18:21 | |
| *** ominakov has quit IRC | 18:21 | |
| yfried | it speaks about a python script. I'd love to see it | 18:22 |
| fungi | i'm busy trying to fix something else at the moment, but the puppet manifests there for the logstash server should show where they're being installed from | 18:22 |
| jeblair | zaro: yes, i can help -- what's up | 18:22 |
| jeblair | fungi, yfried i'll find the scripts and fix the docs | 18:22 |
| yfried | jeblair: tnx | 18:22 |
| *** hashar has joined #openstack-infra | 18:23 | |
| mriedem | fungi: jeblair: does updating a commit message in place with new gerrit reset the review scores? assuming 'yes' | 18:23 |
| fungi | mriedem: i assume it makes a new patchset and everything that implies | 18:23 |
| jeblair | fungi: mriedem yep | 18:23 |
| fungi | but haven't tried it myselg | 18:23 |
| mriedem | k, just making sure, thanks | 18:23 |
| fungi | myself | 18:24 |
| mordred | morning jeblair | 18:24 |
| yfried | so, if I get this - upon finishing the job, jenkins pushes the logs via zmq to the logstash server, together with tags about build/job/etc..? | 18:24 |
| * zaro is afk, bb in 2hrs | 18:24 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Correct log processor script links in docs https://review.openstack.org/91431 | 18:25 |
| fungi | yfried: jenkins uploads the logs to a webserver and sends a zmq message about them which gets interpreted by a gearman client | 18:25 |
| fungi | sort of convoluted | 18:25 |
| jeblair | yfried: the 2 scripts mentioned in the docs are here: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/log_processor/files/ | 18:25 |
| fungi | well, interpreted by a zmq listener which in turn coordinates the indexing via gearman protocol to the cluster | 18:25 |
| *** e0ne has joined #openstack-infra | 18:25 | |
| jeblair | mordred: mornin | 18:26 |
| fungi | yfried: but anyway, jenkins does not send the logs to logstash. it just sticks them on a webserver, then the logstash processing scripts act on the notification and go retrieve them from the same webserver | 18:27 |
| mordred | jeblair: it's wet on the east coast | 18:28 |
| sdague | yes, it is | 18:28 |
| fungi | mordred: indeed. the weather radio woke me up squaking about tornadoes just a few miles north of here | 18:28 |
| yfried | fungi: where is the code to make logstash retrieve the logs? | 18:28 |
| fungi | mordred: better than an alarm clock | 18:28 |
| jeblair | mordred: it is 83 going on 91 at your house. | 18:29 |
| mordred | jeblair: *sob* | 18:29 |
| *** rustlebee has left #openstack-infra | 18:29 | |
| jeblair | and not wet. | 18:29 |
| fungi | yfried: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/log_processor/files/log-gearman-worker.py#n117 | 18:29 |
| sdague | what, only an inch of rain since midnight here :) - http://www.wunderground.com/personal-weather-station/dashboard?ID=KNYPOUGH7#history/s20140430/e20140430/mtoday | 18:32 |
| yfried | fungi: tnx. this might be a stupid question, but what exactly is gearman? I've seen it mentioned, and I now it's designed to help distribute jenkins load, but I'm not too clear | 18:33 |
| fungi | yfried: http://gearman.org/ | 18:33 |
| fungi | yfried: it's for coordinating task distribution | 18:34 |
| *** thomasbiege has joined #openstack-infra | 18:34 | |
| *** aconrad has joined #openstack-infra | 18:35 | |
| fungi | yfried: though we use a pure-python client+server implementation of the gearman protocol called "gear" http://git.openstack.org/cgit/openstack-infra/gear/tree/ | 18:35 |
| yfried | so gearman is the "webserver" waiting to get logs via zmq and telling logstash to parse them? | 18:39 |
| yfried | fungi: ^ | 18:39 |
| fungi | yfried: no... jobs run on a jenkins slave and generate logs. jenkins master pulls logs from the slave and scp's them to a linux server named logs.openstack.org | 18:40 |
| fungi | that linux server serves those raw log files via apache | 18:40 |
| *** arnaud__ has joined #openstack-infra | 18:40 | |
| *** dkliban_afk is now known as dkliban | 18:41 | |
| fungi | separately, when the jenkins master is done scp'ing those logs up to logs.o.o, it also sends a zeromq message about their existence | 18:41 |
| fungi | the logstash cluster has a client subscribed to those zeromq notifications | 18:41 |
| fungi | it then communicates separately to the other logstash indexer cluster workers letting them know there's a log processing task which needs to be done | 18:42 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Adds insecure option to launch_node.py https://review.openstack.org/86690 | 18:42 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Adds insecure option to launch_node.py https://review.openstack.org/86690 | 18:42 |
| fungi | ...via gearman protocol. one of those indexing workers accepts the job and then retrieves the corresponding files from logs.o.o | 18:42 |
| *** gabriel-bezerra has joined #openstack-infra | 18:42 | |
| jeblair | mriedem, fungi: data point -- i updated commit message on https://review.openstack.org/86690 after leaving a -1 and my -1 stuck | 18:43 |
| jeblair | mriedem, fungi: don't know if that's because it was MY -1, or if it would do that for all -1s | 18:43 |
| *** timrc-afk is now known as timrc | 18:44 | |
| yfried | fungi: so the script you sent me (log-gearman-worker) is for which part? | 18:44 |
| *** crc32 has quit IRC | 18:44 | |
| fungi | yfried: that was a link to the class which does that last bit (runs on the indexing cluster workers and downloads the log files from the logs.o.o webserver) | 18:45 |
| *** manishg has joined #openstack-infra | 18:45 | |
| sdague | jeblair: https://review.openstack.org/#/q/status:open+file:%255E.*layout.yaml+NOT+label:Code-Review%253E%253D-2%252Cself,n,z - oh, how I love secondary indexes | 18:45 |
| *** _nadya_ has joined #openstack-infra | 18:45 | |
| mordred | sdague: your blog post on them is amazing | 18:46 |
| sdague | mordred: thanks! | 18:47 |
| mordred | I need to set up one of those for setup.py, setup.cfg and tox.ini in all projects | 18:47 |
| fungi | jeblair: mordred: not sure what's changed, but with the new bare-precise images nodepool nodes are ending at "Exception: Timeout waiting for ssh access" after quite a lot of "INFO nodepool.utils: Password auth exception. Try number X..." messages in the log | 18:47 |
| fungi | the images built successfully, but i wonder if our ssh key changes have screwed something up? | 18:47 |
| *** nati_ueno has quit IRC | 18:47 | |
| *** mrodden1 has joined #openstack-infra | 18:48 | |
| *** nati_ueno has joined #openstack-infra | 18:48 | |
| mordred | fungi: hrm. on both providers? | 18:48 |
| fungi | mordred: yes, which leads me to believe it's courtesy of a recent patch (some time last night or this morning) | 18:49 |
| *** nati_ueno has quit IRC | 18:49 | |
| mordred | fungi: nod | 18:49 |
| mordred | fungi: we were doing something with the ssh_authorized_keys yesterda | 18:49 |
| *** zns_ has quit IRC | 18:49 | |
| mordred | fungi: I'm going to look at nodepool log to see if there's a puppet error there | 18:50 |
| *** mrodden has quit IRC | 18:50 | |
| fungi | good idea. maybe puppet didn't abort but failed to do something | 18:50 |
| gabriel-bezerra | hi, where can I find devstack core reviewers? | 18:50 |
| mordred | yup | 18:50 |
| gabriel-bezerra | I mean, there is no #openstack-devstack channel | 18:50 |
| *** nati_ueno has joined #openstack-infra | 18:51 | |
| fungi | gabriel-bezerra: here and there... #openstack-qa is where a lot of devstack discussions happen | 18:51 |
| morganfainberg | gabriel-bezerra, i think #openstack-qa | 18:51 |
| fungi | gabriel-bezerra: the actual list of people is https://review.openstack.org/#/admin/groups/50,members | 18:51 |
| gabriel-bezerra | ok. thank you. fungi and morganfainberg | 18:52 |
| *** tjones has left #openstack-infra | 18:52 | |
| yfried | fungi: the scripts starts upon recieving zmq msg? | 18:52 |
| *** e0ne has quit IRC | 18:54 | |
| mordred | fungi: I have a patch up to fix this from yesterday | 18:55 |
| fungi | mordred: oh! | 18:55 |
| *** e0ne has joined #openstack-infra | 18:55 | |
| yfried | fungi: how do I get the zmq plugin to my jenkins? should it be available via jenkins plugins page? | 18:56 |
| mordred | fungi: but it looks landed... | 18:56 |
| yfried | fungi: does it need to be DL from the githup repo? | 18:56 |
| mordred | 2014-04-30 17:07:31,207 INFO nodepool.image.build.rax-iad.bare-precise: merr: /Stage[main]/Openstack_project::Users/User::Virtual::Localuser[corvus]/Ssh_authorized_key[jeblair@operational-necessity]: Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/corvus/.ssh/authorized_keys: Permission denied - /home/corvus/.ssh/authorized_key | 18:57 |
| *** zns has joined #openstack-infra | 18:57 | |
| fungi | yfried: i believe it can be installed from the jenkins plugins site. clarkb's our resident expert on this and is on vacation this week. i'm not 100% positive we're using the build on the jenkins plugin site | 18:57 |
| *** ramashri has joined #openstack-infra | 18:57 | |
| *** david-lyle has quit IRC | 18:57 | |
| fungi | mordred: so that's the 0400 vs 0600 perms on teh a_k file | 18:58 |
| *** sdake_ has joined #openstack-infra | 18:59 | |
| *** crc32 has joined #openstack-infra | 18:59 | |
| mordred | fungi: that SHOULD be fixed now | 18:59 |
| mordred | I'm confused | 18:59 |
| *** ramashri has quit IRC | 19:00 | |
| fungi | mordred: i'm manually booting an instance from one of those images now and will log in/poke around | 19:00 |
| mordred | fungi: like, the patch has landed - I can see the corvus_sshkey resource in the log | 19:00 |
| mordred | fungi: good luck - it's not going to have your ssh key | 19:00 |
| mordred | fungi: you may have to log in from a jenkins master as jenkins | 19:00 |
| fungi | mordred: it will because i've told nova boot --file /root/.ssh/authorized_keys=$HOME/.ssh/id_rsa.pub | 19:01 |
| fungi | mordred: but i can also use password login via the web virtual console in rackspace's dashboard | 19:01 |
| yfried | fungi: I'm trying to build a logstash instance for our own ci jobs | 19:01 |
| *** ramashri_ has joined #openstack-infra | 19:01 | |
| notmyname | no meetbot in -meeting? | 19:01 |
| notmyname | jeblair: ^ | 19:01 |
| yfried | fungi: I tried to send a single job log data directly to logstash (via redis) and it was a very slow process, so I'm looking for a more efficient way, trying to learn from your setup | 19:01 |
| notmyname | oh, it's there. just lots of lag | 19:02 |
| fungi | yfried: yes, the complexity in our configuration evolved from multiple iterations of dealing with massive scaling problems | 19:02 |
| jeblair | notmyname: yeah, seems to be lag | 19:02 |
| fungi | yfried: and even so, we're only able to keep about 2 weeks worth of job logs indexed (that's about a terabyte of indexes on each elasticsearch cluster member i believe, of which we currently have 6) | 19:03 |
| *** sdague has quit IRC | 19:03 | |
| *** sdague has joined #openstack-infra | 19:03 | |
| *** mrodden has joined #openstack-infra | 19:07 | |
| *** mrodden1 has quit IRC | 19:07 | |
| yfried | fungi: we don't have that much volume of jobs as the gate does | 19:09 |
| yfried | fungi: and I'm more concerned with stress and prolonging the job, that with storage space | 19:09 |
| *** chandan_kumar has joined #openstack-infra | 19:09 | |
| fungi | yfried: right, the system load and processing speed is why our indexing cluster is 16 rather large virtual machines each with (up to 4?) indexers running on separate cores | 19:10 |
| fungi | i forget the exact count, but it's big | 19:11 |
| fungi | and that's so that we have some hope of having logs indexed within a few minutes of the job completing | 19:11 |
| *** ramashri_ has quit IRC | 19:13 | |
| yfried | fungi: tnx. you helped my a lot. what's your timezone? | 19:14 |
| fungi | yfried: most of the time it's utc-4 or utc-5 | 19:14 |
| fungi | utc-4 at the moment | 19:15 |
| yfried | fungi: that's US time right? too bad. do you know anyone who has similar understanding of logstash in europe timezone? | 19:15 |
| fungi | mordred: it appears that our individual authorized_keys files are 0600 root:root and zero bytes in length | 19:16 |
| fungi | so perms are no longer 0400 at least, but still not owned by our accounts and have no contents | 19:17 |
| fungi | mordred: also, ~jenkins/.ssh/authorized_keys doesn't exist at all | 19:18 |
| mordred | fungi: sigh | 19:20 |
| mordred | fungi: SO - I think ... | 19:20 |
| mordred | the patch to fix the perms only works with existing files | 19:20 |
| mordred | so, now that we've migrated, we should revert that portion of the patch | 19:21 |
| mordred | and I should have just updated the perms with ansible instead | 19:21 |
| fungi | i think most sites using puppet must do kerberos or ldap for login management ;) | 19:21 |
| mordred | because ssh_auth_key worked in nodepool | 19:21 |
| mordred | before the patch | 19:21 |
| mordred | so it worked without the patch on a clean machine | 19:21 |
| *** e0ne has quit IRC | 19:22 | |
| fungi | mordred: do you think the file permissions change could also be responsible for the missing ~jenkins/.ssh/authorized_keys file? or is that likely a side effect of some other related key management change? | 19:25 |
| *** rloo has joined #openstack-infra | 19:25 | |
| *** nati_ueno has quit IRC | 19:25 | |
| mordred | yes | 19:26 |
| mordred | I thnk it's the file perms | 19:26 |
| mordred | because it introduced a file resource with no content | 19:27 |
| fungi | because the manifestation differs. for our accounts it's empty authorized_keys files owned by root. for the jenkins user it's a missing file entirely | 19:27 |
| fungi | mordred: are you drafting the rip-out for that bit, or should i? | 19:28 |
| rloo | hi. I wanted to get python-ironicclient dev docs up on docs.openstack.org/developer/. I added jobs to jenkins/zuul and the docs are on docs-draft.openstack.org. What else do I need to do? | 19:29 |
| jeblair | rloo: the change to add the new jobs has merged? | 19:30 |
| rloo | jeblair: yes. https://review.openstack.org/#/c/86390/ | 19:30 |
| rloo | jeblair: and i can see them on docs-draft, eg http://docs-draft.openstack.org/18/72418/11/check/gate-python-ironicclient-docs/747b1af/doc/build/html/ | 19:30 |
| rloo | jeblair: what copies the stuff from docs-draft to docs.openstack.org/developer? | 19:31 |
| jeblair | rloo: it looks like client docs are only published on release | 19:31 |
| jeblair | rloo: so it should happen the next time a release is tagged | 19:31 |
| rloo | jeblair: hmm. is that how other client docs are done? I copy/pasted from others. | 19:31 |
| jeblair | rloo: yes. i believe the intention is to avoid confusing users by publishing documentation for unreleased software | 19:32 |
| rloo | jeblair: I assume there's a reason to do things that way. Ok, I'll discuss with deva. thx! | 19:32 |
| jeblair | rloo: and the clients release more frequently than the servers | 19:32 |
| *** mfer has quit IRC | 19:32 | |
| *** david-lyle has joined #openstack-infra | 19:32 | |
| rloo | jeblair: thx again! | 19:33 |
| jeblair | rloo: if you are going to release soon, i'd prefer to just wait. if you are not going to release for quite a while and otherwise have no reason to, we can try to run the job manually for the most recently released version. | 19:33 |
| jeblair | rloo: you're welcome. let me know what you learn. | 19:33 |
| devananda | rloo: hm? | 19:33 |
| rloo | devananda: didn't realize you were here. ^^. our client docs won't show up until we do a release. | 19:34 |
| pabelanger | So, how do global requirement gerrit reviews get created across all the projects? Is that an automated process? | 19:34 |
| devananda | oh - sure. i'm happy to tag a release. lemme see what the changes are | 19:34 |
| fungi | pabelanger: yes, it's a triggered job which runs a script which proposes them | 19:34 |
| fungi | pabelanger: triggered off a change merging to the requirements repo | 19:35 |
| rloo | devananda: thx. | 19:35 |
| fungi | pabelanger: so look in the post pipeline for the requirements project | 19:35 |
| *** ihrachyshka has joined #openstack-infra | 19:35 | |
| dkranz | fungi: In https://review.openstack.org/#/c/80620/7, there is no indication in the check/verify matrix of who gave the to +2/+A (I was one of them). Is that new behavior or I just never noticed it? | 19:36 |
| dkranz | fungi: two +2/+A | 19:37 |
| fungi | dkranz: i think that's a side effect of the approve label going away | 19:38 |
| dkranz | fungi: Is that bug or a feature? | 19:38 |
| *** ihrachyshka has quit IRC | 19:38 | |
| *** ihrachyshka has joined #openstack-infra | 19:38 | |
| jeblair | dkranz: misfeature, i think. you can get the info by reading the review comments though. | 19:39 |
| dkranz | fungi: I would think it should show a +2 for the Approver if it is not showing Approve column | 19:39 |
| jeblair | David KranzApr 23 10:18 AM | 19:39 |
| jeblair | Patch Set 7: Approved | 19:39 |
| dkranz | jeblair: Yes, I saw that. That's how I remembered it was me :) | 19:39 |
| *** thomasbiege has quit IRC | 19:39 | |
| sdague | https://review.openstack.org/#/c/90812/ - mordred / SergeyLukjanov / morganfainberg - I actually think this should be in d-g, and not in devstack | 19:39 |
| fungi | dkranz: i suspect new gerrit is unable to interpret your vote since it includes a vote on a label it doesn't think exists, so it's hiding you in that table | 19:39 |
| jeblair | sdague: can we make running those under apache the default? | 19:40 |
| dkranz | fungi: So are you saying this will only happen when looking at reviews from before the change? | 19:40 |
| fungi | dkranz: yes | 19:40 |
| sdague | jeblair: I don't know | 19:41 |
| dkranz | fungi: ok, then that is not so bad. | 19:41 |
| fungi | dkranz: reviews approved after the upgrade will be workflow +1 instead | 19:41 |
| dkranz | fungi: thansk | 19:41 |
| sdague | morganfainberg: opinions on that? | 19:41 |
| *** doddstack has joined #openstack-infra | 19:41 | |
| jeblair | sdague: i think it's better for d-g to run as close to the default devstack as possible -- a big part of the goal is to make sure we can safely say to anyone "just run devstack, it will work" | 19:41 |
| fungi | dkranz: we could consider retro-fixing the old changes in the db, but gerrit does some very funny things with mysql auto-timestamp fields so you have to be extremely careful to reset them to their current values. otherwise the chronology goes out the window | 19:42 |
| sdague | jeblair: sure, but the proposal on the devstack side wasn't to actually change the default. It sounded like right now morganfainberg wanted a switch | 19:42 |
| fungi | dkranz: and i'm so far unconvinced it's worth the risk/effort | 19:43 |
| portante | does anybody consume the coverage report jobs output? | 19:43 |
| sdague | unrelated - https://review.openstack.org/#/c/87677/ - the heat team would really like to be able to work on heat adds for grenade, which they need this for | 19:43 |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Set Alpha Channel for better looking favicon https://review.openstack.org/91441 | 19:43 |
| *** chandan_kumar has quit IRC | 19:43 | |
| pabelanger | fungi, still looking but the hook script for the repo is added into zuul? | 19:44 |
| dkranz | fungi: I don't think it is worth it either. I just noticed the behavior and did not know it was transient. | 19:44 |
| fungi | pabelanger: yeah, it's one of the jenkins slave scripts in our config repo | 19:44 |
| pabelanger | fungi, okay, think I see it | 19:45 |
| fungi | pabelanger: there's a job in the post pipeline of the requirements project in zuul's layout which should correspond to something in our jjb config which ought to lead you to a builder macro which calls that slave script which inspects the projects.txt in the requirements repo and then proposes changes to the projects listed there | 19:45 |
| *** ociuhandu has quit IRC | 19:45 | |
| *** mfer has joined #openstack-infra | 19:45 | |
| fungi | rube goldberg would be proud | 19:45 |
| pabelanger | Ya, see it now: propose-requirements-updates | 19:47 |
| pabelanger | okay cool. Ya, I want to setup a periodic gate for something and liked the way the requirements stuff works. Think I'll use the same logic for creating new reviews | 19:48 |
| *** mattymo|home has quit IRC | 19:48 | |
| jeblair | sdague: https://review.openstack.org/#/c/88289/ | 19:49 |
| *** talluri has quit IRC | 19:51 | |
| *** talluri has joined #openstack-infra | 19:51 | |
| sdague | jeblair: one minor issue I see in it | 19:53 |
| *** _nadya_ has quit IRC | 19:53 | |
| sdague | I'll admit my brain is not fully wrapped around the -2 jobs, so you'll have to tell me if those are right or not | 19:54 |
| sdague | conceptually +1 | 19:54 |
| sdague | review feedback reported | 19:54 |
| *** talluri has quit IRC | 19:54 | |
| *** mwagner_lap has quit IRC | 19:55 | |
| *** talluri has joined #openstack-infra | 19:55 | |
| *** rcarrillocruz has joined #openstack-infra | 19:56 | |
| *** rcarrill` has quit IRC | 19:58 | |
| jeblair | sdague: nice catch. as for -2; i'm counting on the layout check to tell us if they don't exist. it didn't complain so i assume they do. i belive the proposed config will run them on stable branches only. | 19:59 |
| portante | coverage, coverage, ... anyone, anyone, ... bueller, bueller? | 19:59 |
| portante | ;) | 19:59 |
| *** andreykurilin_ has joined #openstack-infra | 20:00 | |
| jeblair | sdague: http://logs.openstack.org/89/88289/5/check/gate-config-layout/9cfde90/console.html.gz | 20:00 |
| jeblair | <Job check-tempest-dsvm-neutron-2> ^(stable/).*$ | 20:00 |
| jeblair | so that one will only run on stable branches | 20:00 |
| jeblair | <Job check-tempest-dsvm-neutron-pg-2> <Job check-tempest-dsvm-neutron-full-2> <Job check-tempest-dsvm-neutron-pg-full-2> run on all branches | 20:00 |
| *** nati_ueno has joined #openstack-infra | 20:00 | |
| sdague | I think we want them not on stable branches? | 20:01 |
| portante | I'd like to be able to access the results of the coverage jobs | 20:01 |
| sdague | salv-orlando: still up? | 20:01 |
| *** Ryan_Lane1 has joined #openstack-infra | 20:01 | |
| jeblair | portante: take the sha of the commit you're interested in; eg HEAD is 59a639c116eba2211dad722b287f9a8675a5155d | 20:02 |
| jeblair | portante: then go to http://logs.openstack.org/59/59a639c116eba2211dad722b287f9a8675a5155d/ | 20:02 |
| *** rloo has quit IRC | 20:03 | |
| *** markmcclain has joined #openstack-infra | 20:03 | |
| *** Ryan_Lane has quit IRC | 20:03 | |
| jeblair | (i think there's a git subcommand floating around to do that automatically) | 20:03 |
| Ajaeger | jeblair: git os-job | 20:04 |
| portante | jeblair: thanks | 20:04 |
| jeblair | that's it | 20:04 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Stop over-managing user SSH keyfiles https://review.openstack.org/91449 | 20:04 |
| *** nati_ueno has quit IRC | 20:04 | |
| *** hogepodge has quit IRC | 20:05 | |
| fungi | mordred: jeblair: ^ | 20:05 |
| jogo | top of gate is waiting for docs, pep8 and py27 jobs http://status.openstack.org/zuul/ | 20:05 |
| *** Ajaeger has quit IRC | 20:05 | |
| fungi | jogo: yep. fixing | 20:05 |
| jogo | fungi: awesome sauce | 20:05 |
| *** hogepodge has joined #openstack-infra | 20:06 | |
| jeblair | fungi: does anything pass in "old_keys"? | 20:06 |
| jogo | sdague: have you seen this yet: http://logs.openstack.org/46/84446/24/gate/gate-tempest-dsvm-full/84cd8ad/logs/devstacklog.txt.gz#_2014-04-29_23_18_16_856 | 20:07 |
| fungi | jeblair: nope. not that my git grep in the config repo found | 20:07 |
| jogo | fedoraproject is giving us 403s | 20:07 |
| jeblair | fungi: i think old_keys might be unrelated to this -- i think it's to support key rollover | 20:07 |
| fungi | jeblair: oh | 20:07 |
| fungi | i'll put it back | 20:07 |
| sdague | jogo: yes, that was supposed to be cached in nodepool though | 20:07 |
| jeblair | fungi: cool, rest looks good | 20:07 |
| sdague | are our nodepool images super old? | 20:08 |
| jogo | sdague: oh :/ is there a bug filed for this | 20:08 |
| fungi | jeblair: i had originally asked mordred why it was there and he said it was transitional, but we might have been talking past each other | 20:08 |
| *** MikeLeman has joined #openstack-infra | 20:08 | |
| sdague | jogo: I don't think so | 20:08 |
| * jogo files an infra bug and a e-r patch for it | 20:09 | |
| sdague | I had thought I had that change in there long enough to get into the nodepool image, but now am realizing we might be working off of really old nodepool images now | 20:09 |
| jeblair | sdague: hp is so broken we only get a new image out of them every few weeks | 20:10 |
| jogo | sdague: ohh HP :) don't you like your new corporate overlord? | 20:10 |
| sdague | heh | 20:11 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Stop over-managing user SSH keyfiles https://review.openstack.org/91449 | 20:11 |
| sdague | jeblair: hmmm... | 20:11 |
| jeblair | jogo: good point, it's all sdague's fault now. it used to be yours but now it's his. ;) | 20:11 |
| sdague | jeblair: ok, well followup question | 20:12 |
| jeblair | sdague: apparently hpcloud1.1 is better at this, but we're on the fence about whether it's better at spinning up nodes once we actually have images. | 20:12 |
| jeblair | sdague: morded was looking into that. | 20:12 |
| sdague | the issue is fedora download sites are terrible | 20:12 |
| sdague | in their uptime | 20:12 |
| fungi | jeblair: mordred: as soon as we approve https://review.openstack.org/91449 i'll force merge it (we don't currently have available bare-precise slaves to test it for style checks) and then build new images asap | 20:12 |
| sdague | however, if we could put the image somewhere we were more sure we could pull from | 20:12 |
| jogo | sdague jeblair: https://bugs.launchpad.net/openstack-ci/+bug/1314786 | 20:12 |
| uvirtbot | Launchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not cached" [Undecided,New] | 20:12 |
| sdague | maybe we could stick it there instead | 20:12 |
| jeblair | fungi: lgtm; i'll let you aprv/subm | 20:13 |
| sdague | because otherwise updating some of these base images for testing will take months to ensure that nodepool images have them | 20:13 |
| jeblair | sdague: the right answer is to have clouds that work | 20:13 |
| sdague | sure | 20:13 |
| openstackgerrit | A change was merged to openstack-infra/config: Stop over-managing user SSH keyfiles https://review.openstack.org/91449 | 20:13 |
| sdague | I can't fix that part :) | 20:13 |
| jeblair | we can fix it -- move to hpcloud1.1, and that's where we should put our effort | 20:14 |
| sdague | ok | 20:14 |
| fungi | i did at least test the patch on a manually-launched node from one of the broken images to make sure that it dtrt | 20:14 |
| jogo | jeblair: what happened to that, I thought that already happened | 20:14 |
| jeblair | fungi: you have high standards | 20:14 |
| jogo | or at least you previously mentioned it | 20:14 |
| sdague | jeblair: so what needs to happen to get us there? | 20:14 |
| jeblair | jogo: we're 1/3 of the way there, ask mordred for progress report | 20:14 |
| jogo | mordred: ^ | 20:15 |
| fungi | and it does in fact seem to be capable of adding the keyfiles and ssh dires automatically, and appears to calculate its dependence on the user objects as well | 20:15 |
| *** freyes_ has joined #openstack-infra | 20:15 | |
| fungi | s/dires/dirs/ | 20:15 |
| *** freyes_ has quit IRC | 20:15 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1314786 https://review.openstack.org/91453 | 20:16 |
| uvirtbot | Launchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not cached" [Undecided,New] https://launchpad.net/bugs/1314786 | 20:16 |
| jogo | sdague: ^ e-r patch | 20:17 |
| *** Ryan_Lane1 has quit IRC | 20:17 | |
| sdague | jogo: should be tags:console | 20:18 |
| sdague | because we'll start seeing this in grenade at some point | 20:18 |
| jogo | ack, found another manifestion as well | 20:18 |
| jogo | http://logs.openstack.org/47/90247/1/gate/gate-tempest-dsvm-neutron/813bb88/logs/devstacklog.txt.gz#_2014-04-30_00_04_47_103 | 20:18 |
| jogo | sdague: (message:"awaiting response... 403 Forbidden" OR message:"Unable to establish SSL connection.") AND tags:"console" | 20:20 |
| jogo | I am not sure how to include fedora in the query | 20:20 |
| sdague | I don't think you can | 20:21 |
| jogo | with just the console tag it may be a pretty broad query, but we will see | 20:21 |
| *** dprince has quit IRC | 20:21 | |
| sdague | any egregious hits on that which don't look like this problem? | 20:21 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1314786 https://review.openstack.org/91453 | 20:21 |
| uvirtbot | Launchpad bug 1314786 in openstack-ci "/opt/stack/new/devstack/files/Fedora-x86_64-20-20131211.1-sda.qcow2 is not cached" [Undecided,New] https://launchpad.net/bugs/1314786 | 20:21 |
| jogo | where this problem involves fedora | 20:21 |
| jogo | mirror | 20:22 |
| sdague | correct | 20:22 |
| gabriel-bezerra | something is definitely wrong with zuul. look at how long all those builds are taking to run | 20:22 |
| gabriel-bezerra | more than 3h | 20:22 |
| fungi | gabriel-bezerra: yes, it will be working again soon | 20:22 |
| jogo | sdague: anyway next question: what are your thoughts on adding neutron grenade logs to e-r? | 20:22 |
| sdague | to e-s | 20:23 |
| jogo | err yes | 20:23 |
| sdague | yes, we should do that | 20:23 |
| fungi | #status alert the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc) | 20:23 |
| openstackstatus | fungi: sending alert | 20:23 |
| * jogo preps a patch | 20:23 | |
| *** _nadya_ has joined #openstack-infra | 20:24 | |
| morganfainberg | sdague, here now | 20:24 |
| -openstackstatus- NOTICE: the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc) | 20:25 | |
| morganfainberg | sdague, sorry was at lunch | 20:25 |
| *** ChanServ changes topic to "the gate is backed up due to broken nodepool images, fix in progress (eta 22:00 utc)" | 20:25 | |
| morganfainberg | sdague, i'm fine with it being in either ds-g or in devstack | 20:25 |
| *** SumitNaiksatam has quit IRC | 20:25 | |
| morganfainberg | sdague, both work for me. | 20:25 |
| sdague | morganfainberg: so the question is about defaults | 20:25 |
| *** nati_ueno has joined #openstack-infra | 20:25 | |
| morganfainberg | sdague, i don't want to change any defaults at this point | 20:25 |
| morganfainberg | sdague, eventually we might make keystone default to an apache deploy, but we want to test it for sure | 20:26 |
| sdague | ok, so if we're not changing defaults, I think d-g is the right place for this | 20:26 |
| *** mrodden has quit IRC | 20:26 | |
| morganfainberg | sdague, my goal is to make a new check (non-vote) job to start with, eventually gate on it | 20:26 |
| *** Ryan_Lane has joined #openstack-infra | 20:26 | |
| sdague | morganfainberg: ok, so you're going to be at the test matrix session right? :) | 20:26 |
| morganfainberg | sdague, was planning on it. haven't done my schedule yet | 20:26 |
| morganfainberg | sdague :) | 20:26 |
| openstackstatus | fungi: finished sending alert | 20:26 |
| sdague | because we can't just multiply out another keystone config independant of all the other configs | 20:27 |
| sdague | at least not across everything | 20:27 |
| *** mrodden has joined #openstack-infra | 20:27 | |
| morganfainberg | sdague, correct. i was planning on making this a keystone-only check/gate job for now | 20:27 |
| sdague | ok, so lets play that game to get started. You might also want to only include keystone in the apache services list then | 20:28 |
| morganfainberg | sdague, ++ was just going to suggest isolating this to keystone for this change | 20:28 |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Handle file renames in diff view https://review.openstack.org/91458 | 20:28 |
| morganfainberg | sdague, or well say that was my plan in this case. | 20:28 |
| morganfainberg | eventually i think we should have a mod_wsgi based check for anything that can run under mod_wsgi... but different argument different time | 20:29 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Add grenade neutron logs to ElasticSearch https://review.openstack.org/91459 | 20:29 |
| morganfainberg | sdague, thanks for looking at it :) | 20:29 |
| jeblair | yeah, if we're diverging from the default then we should put a switch in d-g -- but when it becomes the defacto default in d-g, or even the recommended way to deploy in devstack, it should become the default in devstack and eventually be removed from d-g | 20:30 |
| morganfainberg | jeblair, ++ 100% agree | 20:30 |
| sdague | jeblair: agreed | 20:30 |
| *** eharney has quit IRC | 20:33 | |
| *** aconrad has quit IRC | 20:35 | |
| *** maxbit has quit IRC | 20:37 | |
| openstackgerrit | Antoine Musso proposed a change to stackforge/python-jenkins: Wrap tests and linting with tox https://review.openstack.org/89521 | 20:37 |
| sdague | anyone know who owns the ops tracks at design summit? | 20:37 |
| fungi | sdague: the user committee i believe | 20:38 |
| hashar | jeblair: will you have anytime this week to review/merge the addition of tox to stackforge/python-jenkins ? That is preventing us from adding triggers in Zuul for that repo :-) | 20:38 |
| fungi | sdague: might ask fifieldt and/or reed when they're around | 20:39 |
| *** sdake_ has quit IRC | 20:39 | |
| sdague | fungi: thanks | 20:39 |
| *** ociuhandu has joined #openstack-infra | 20:39 | |
| jeblair | hashar: i know! i just saw the review to config that added the jobs. i've gone from 96 reviews to 37 so far today. i'm getting close. i'll add you and mgagne to the group for python-jenkins. | 20:42 |
| hashar | jeblair: take your time. I know you are a busy guy :] | 20:42 |
| *** jcoufal has quit IRC | 20:42 | |
| jeblair | hashar: you should have +2 there now | 20:42 |
| ArxCruz | anteaya: jeblair hi, after I upgrade my zuul and layout.yaml today, I start to subscribe in all changes (although I'm not voting) is that normal from now ? | 20:42 |
| hashar | jeblair: you are awesome :] Can you add Khai Do as well? he is very active on jjb | 20:42 |
| *** Ryan_Lane has quit IRC | 20:43 | |
| jeblair | zaro: done ^ | 20:43 |
| hashar | unless he objects of course | 20:43 |
| jeblair | heh, he can remove himself ;) | 20:43 |
| jeblair | ArxCruz: i'm not sure i understand the question | 20:43 |
| jeblair | ArxCruz: (specifically, i don't know what you mean by subscribe) | 20:43 |
| ArxCruz | jeblair: https://review.openstack.org/#/c/90607/ | 20:44 |
| ArxCruz | IBM Power KVM is in the review list | 20:44 |
| ArxCruz | my layout.yaml have gerrit: { } as you told me to do yesterday | 20:45 |
| ArxCruz | this is making us receive a lot of emails for every change :) | 20:45 |
| *** lcostantino has quit IRC | 20:46 | |
| fungi | ArxCruz: it looks like IBM PowerKVM Testing reported on that change | 20:46 |
| fungi | at 15:24 | 20:46 |
| *** adalbas has quit IRC | 20:47 | |
| *** aconrad has joined #openstack-infra | 20:47 | |
| ArxCruz | fungi: yeah, but ibm powerkvm reported before, but never was added in the reviewer list | 20:47 |
| fungi | ArxCruz: odd, i thought any time an account commented on a change it was automatically added to the list of reviewers for that change | 20:48 |
| *** etoews has joined #openstack-infra | 20:48 | |
| ArxCruz | fungi: nope, that's why I'm asking if that's the new behavior | 20:48 |
| *** mbacchi has quit IRC | 20:48 | |
| fungi | ArxCruz: it's possible you were managing to not end up on the reviewers list for changes previously (i'm not sure how that was happening but i assume it must have been) and gerrit 2.8 perhaps behaves differently in that regard than 2.4 did | 20:49 |
| ArxCruz | krtaylor: ^ | 20:49 |
| ArxCruz | fungi: that's what I want to confirm :) | 20:49 |
| jeblair | ArxCruz: my understanding was the same as fungi's. | 20:50 |
| fungi | the other possibility is that the tweaks we had to do to zuul to support non-voting changes in the 2.8 api altered that behavior subtly? | 20:50 |
| ArxCruz | well, i'm using latest zuul | 20:51 |
| fungi | right, zuul got some patches when we upgraded gerrit, related to supporting newer gerrit | 20:51 |
| *** jcoufal has joined #openstack-infra | 20:52 | |
| *** jcoufal has quit IRC | 20:53 | |
| *** Ryan_Lane has joined #openstack-infra | 20:55 | |
| *** zns has quit IRC | 20:56 | |
| ArxCruz | well, I can't investigate it right now, I'm trying to make puppetboard works correctly here, the report are not appearing | 20:57 |
| ArxCruz | btw if someone have experience on it ... ;) | 20:57 |
| *** marcoemorais has quit IRC | 20:58 | |
| *** zns has joined #openstack-infra | 20:58 | |
| *** arnaud__ has quit IRC | 20:59 | |
| *** ArxCruz has quit IRC | 21:00 | |
| *** pdmars has quit IRC | 21:01 | |
| morganfainberg | pleia2, i'm looking at how to fix reviewday to handle the new WIP mode, just noticed WIP patches floated to the top again. | 21:03 |
| zaro | ohh cool, python-jenkins made it into gerrit. | 21:03 |
| *** _nadya_ has quit IRC | 21:03 | |
| *** sdake_ has joined #openstack-infra | 21:04 | |
| fungi | okay, one of the bare-precise image builds has completed so far (hpcloud-az3). others are close behind i think | 21:04 |
| fungi | 10 nodes building there so far | 21:04 |
| *** thomasem has quit IRC | 21:04 | |
| *** etoews has quit IRC | 21:05 | |
| *** thomasem has joined #openstack-infra | 21:05 | |
| *** etoews has joined #openstack-infra | 21:05 | |
| fungi | 15 | 21:06 |
| *** dkliban is now known as dkliban_afk | 21:06 | |
| boris-42 | jeblair hi | 21:06 |
| *** aconrad has quit IRC | 21:10 | |
| *** thomasem has quit IRC | 21:10 | |
| *** rcarrill` has joined #openstack-infra | 21:10 | |
| fungi | and some in use now | 21:10 |
| *** rcarrillocruz has quit IRC | 21:11 | |
| fungi | rax-ord has a ready image and is building nodes now as well | 21:11 |
| *** markmcclain has quit IRC | 21:12 | |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Handle binary files in diffstat https://review.openstack.org/91467 | 21:12 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/devstack-gate: Allow setting services up under apache mod_wsgi https://review.openstack.org/90812 | 21:12 |
| *** marcoemorais has joined #openstack-infra | 21:13 | |
| jeblair | intervals between fixing gertty for changes it can't handle are increasing! that was 50 minutes of nonstop reviewing! | 21:13 |
| jeblair | boris-42: hello | 21:14 |
| boris-42 | jeblair so about Rally checks | 21:14 |
| boris-42 | jeblair could I introduce you a bit? | 21:15 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/devstack-gate: Allow setting services up under apache mod_wsgi https://review.openstack.org/90812 | 21:15 |
| jeblair | boris-42: i'm personally interested and excited about it, but my comment wasn't based on whether i thought rally was a good idea. it's about how we introduce new jobs like that. | 21:16 |
| jeblair | boris-42: i think what you are proposing should start out in the experimental pipeline so you can demonstrate that it works, and how it works | 21:16 |
| boris-42 | jeblair so actually we already demonstrated in rally gate | 21:17 |
| boris-42 | jeblair basically in rally we are going to use it to check that all benchmarks works | 21:17 |
| boris-42 | jeblair as in keystone check is already merged | 21:18 |
| boris-42 | jeblair let me just show | 21:18 |
| boris-42 | jeblair you are putting here https://review.openstack.org/#/c/90405/3/rally-scenarios/keystone.yaml task configuration for rally | 21:18 |
| *** aconrad has joined #openstack-infra | 21:18 | |
| boris-42 | jeblair it can be one or more benchmarks with any load in any enviroment | 21:18 |
| boris-42 | jeblair as a result you are getting all logs collected (dsvm job) + this pretty graphic http://logs.openstack.org/05/90405/3/check/check-rally-dsvm-keystone/81a809d/rally-plot/results.html.gz | 21:19 |
| *** markwash has quit IRC | 21:19 | |
| *** etoews has quit IRC | 21:19 | |
| boris-42 | jeblair ^ that shows that something is not ok with this thing under load | 21:19 |
| boris-42 | jeblair 0 on graph == exception | 21:19 |
| boris-42 | jeblair so there are 2 use cases | 21:19 |
| fungi | confirmed i could log into one of the fresh bare-precise nodes and here's a job which ran to completion on one as well... https://jenkins05.openstack.org/job/gate-ironic-docs/294/console | 21:20 |
| boris-42 | default task that will allow us to track performance of major functionallity | 21:20 |
| fungi | so i think everything should start catching back up shortly | 21:20 |
| boris-42 | jeblair and actually working on performance | 21:20 |
| fungi | i'm going to take a break for dinner and hopefully clear the status alert when i get back | 21:20 |
| boris-42 | jeblair e.g. 1 patch just changes project_name.yaml and measure gates | 21:20 |
| boris-42 | jeblair probably one time recheck to ensure | 21:21 |
| boris-42 | jeblair and then your code with same content of project_name.yaml | 21:21 |
| boris-42 | jeblair and probably recheck | 21:21 |
| boris-42 | jeblair so we are getting different numbers and able to check does current patch improves performance or not | 21:21 |
| boris-42 | jeblair in future we are going to add normalization of benchmark results + comparison with last merged patch performance | 21:22 |
| boris-42 | jeblair so it will be even more usefully and probably we will be able to make +1/-1 decision depending of changes in perfroamnce | 21:22 |
| *** dkranz has quit IRC | 21:22 | |
| *** eharney has joined #openstack-infra | 21:24 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Create tripleo-specs git repo https://review.openstack.org/89691 | 21:24 |
| zaro | jeblair: so, hope you have time now. about gear ssl. | 21:27 |
| jeblair | zaro: yeah, what do you need? | 21:27 |
| zaro | jeblair: i have setup a vm and i'm trying to test gear ssl by running server, worker, and client all on same vm | 21:27 |
| zaro | jeblair: used this website as instructions to create my certs: http://forum.synology.com/wiki/index.php/How_to_generate_custom_SSL_certificates | 21:28 |
| *** dangers is now known as dangers_away | 21:28 | |
| zaro | jeblair: this is how i start the server: geard -d --ssl-ca=/home/ubuntu/ca.crt --ssl-cert=/home/ubuntu/server.crt --ssl-key=/home/ubuntu/server.key | 21:29 |
| zaro | jeblair: this is my worker connection setting: worker.addServer('localhost',4730,'/home/ubuntu/server.key','/home/ubuntu/server.crt','/home/ubuntu/ca.crt') | 21:30 |
| jeblair | zaro: the worker should get its own cert | 21:30 |
| zaro | jeblair: why? it's on the same machine. | 21:30 |
| *** MikeLeman has quit IRC | 21:31 | |
| *** rlandy is now known as rlandy|afk | 21:31 | |
| *** jgrimm has quit IRC | 21:31 | |
| jeblair | zaro: it's part of how the gear acls work -- they are based on the cn in the cert, so each worker/client should get their own | 21:31 |
| lifeless | jeblair: https://review.openstack.org/#/c/89691/ - are new-project things still once a week ? | 21:32 |
| jeblair | lifeless: no | 21:32 |
| lifeless | jeblair: yay :) | 21:32 |
| jeblair | indeed :) | 21:32 |
| lifeless | (unless of ourese they are now once a month, in which case sad) | 21:32 |
| zaro | jeblair: sorry, i'm newbie to this, not real sure what to do now so i'm gonna ask real simple questions. | 21:33 |
| zaro | jeblair: do you mean use the same instructions to create a seperate set of certs with the same cn name? | 21:33 |
| *** hashar has quit IRC | 21:33 | |
| jeblair | zaro: here's what i did in my testing: http://paste.openstack.org/show/78082/ | 21:34 |
| jeblair | zaro: so yes -- using the commands i supplied, repeat the last two for each client you want to connect. i did it once with CLIENT='client', and another with 'worker' and another with 'server'. | 21:35 |
| jeblair | zaro: then i used the 'server' key for geard, the 'worker' key for my test worker, and the 'client' key when connecting with openssl on the command line. | 21:35 |
| zaro | jeblair: ok. i will try that. thanks. | 21:36 |
| jeblair | zaro: then i used this acl file with geard: http://paste.openstack.org/show/78085/ | 21:37 |
| *** yamahata has quit IRC | 21:37 | |
| jeblair | zaro: (the [worker] and [client] sections there refer to the certificate names above (they don't mean "any worker", the mean "whoever connects with the cert named 'worker'") | 21:37 |
| *** yamahata has joined #openstack-infra | 21:40 | |
| *** rfolco has quit IRC | 21:40 | |
| openstackgerrit | A change was merged to stackforge/gertty: Properly register password from prompt https://review.openstack.org/91392 | 21:41 |
| openstackgerrit | A change was merged to stackforge/gertty: Normalize URL to ends with a slash https://review.openstack.org/91393 | 21:41 |
| *** mriedem has quit IRC | 21:41 | |
| openstackgerrit | A change was merged to stackforge/gertty: Fix editing inline comments https://review.openstack.org/91410 | 21:42 |
| openstackgerrit | A change was merged to stackforge/gertty: Handle "No newline at end of file" and add --no-sync option https://review.openstack.org/91418 | 21:42 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: 100% test coverage https://review.openstack.org/90500 | 21:42 |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Support excludes when recursively processing paths https://review.openstack.org/68895 | 21:42 |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Added recursive option https://review.openstack.org/85106 | 21:42 |
| openstackgerrit | A change was merged to stackforge/gertty: Handle file renames in diff view https://review.openstack.org/91458 | 21:43 |
| openstackgerrit | A change was merged to stackforge/gertty: Handle binary files in diffstat https://review.openstack.org/91467 | 21:43 |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add .gitignore https://review.openstack.org/90188 | 21:46 |
| *** markwash has joined #openstack-infra | 21:47 | |
| *** rhsu has joined #openstack-infra | 21:50 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:54 | |
| *** shardy is now known as shardy_afk | 22:01 | |
| zaro | jeblair: which file corresponds to the 'ssl-ca'? the .crt, .csr or .key? | 22:03 |
| *** aconrad has quit IRC | 22:04 | |
| *** yidclare has quit IRC | 22:05 | |
| jeblair | zaro: for the purpose of configuring gearman, ca.crt is the one you want. | 22:09 |
| salv-orlando | sdague: if you still need me I'm here | 22:09 |
| jeblair | zaro: (your geard command line from earlier looked right) | 22:09 |
| *** gondoi is now known as zz_gondoi | 22:09 | |
| sdague | salv-orlando: I think there was just a question of intent on the full patch | 22:09 |
| sdague | because it made some of the jobs vote on stable only | 22:10 |
| salv-orlando | ah I see the intent is to run smoke jobs on stable only, the full job anywhere but voting only on master | 22:11 |
| salv-orlando | not sure if my patch implements that intent however | 22:11 |
| jeblair | sdague: thoughts on https://review.openstack.org/#/c/89493/ and possible interaction with test_matrix.py? | 22:12 |
| zaro | jeblair: i got a connection! thanks. | 22:12 |
| jeblair | zaro: yaay! | 22:13 |
| mattoliverau | Morning all | 22:14 |
| *** nati_ueno has quit IRC | 22:15 | |
| *** aconrad has joined #openstack-infra | 22:16 | |
| fungi | awesome--i've not missed much | 22:18 |
| zaro | fungi: you might want to pickup this bug? https://bugs.launchpad.net/openstack-ci/+bug/1248794 | 22:19 |
| uvirtbot | Launchpad bug 1248794 in openstack-ci "Allow core reviewers restore auto-abandoned reviews" [Medium,In progress] | 22:19 |
| *** ihrachyshka has quit IRC | 22:21 | |
| fungi | i see fairly many bare-precise nodes running jobs now, but no obvious movement in zuul status | 22:21 |
| openstackgerrit | Alex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript https://review.openstack.org/90507 | 22:22 |
| openstackgerrit | Alex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript https://review.openstack.org/90507 | 22:24 |
| fungi | yeah, something's not right. bare-precise-rax-iad-3912914.slave.openstack.org is running gate-nova-pep8 for 91025,1 but the zuul status page says that's still queued | 22:24 |
| *** atiwari has quit IRC | 22:25 | |
| fungi | it ran in the gate pipeline and completed reporting success... | 22:26 |
| fungi | and then nodepoold marked it for delete | 22:26 |
| fungi | huh, now zuul shows the same job running, no longer queued, but on bare-precise-rax-ord-3913855 | 22:27 |
| *** smarcet has quit IRC | 22:27 | |
| fungi | ahh, i bet the other was queued from a gate reset | 22:28 |
| *** nati_ueno has joined #openstack-infra | 22:28 | |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/config: Clean up some variable references https://review.openstack.org/88475 | 22:28 |
| fungi | and i walked in at just the wrong moment :/ | 22:28 |
| jesusaurus | can i get some eyes on https://review.openstack.org/#/c/89708/ ? | 22:29 |
| *** bknudson has quit IRC | 22:29 | |
| *** mfer has quit IRC | 22:29 | |
| *** yidclare has joined #openstack-infra | 22:30 | |
| fungi | jesusaurus: so 89708 mostly moves jenkins slave scripts out of the jenkins module and into the openstack_project module? | 22:30 |
| jesusaurus | yeah | 22:31 |
| fungi | oh, and a lot of the manifest too actually | 22:31 |
| openstackgerrit | A change was merged to openstack-infra/config: Create bash8 project in openstack-dev https://review.openstack.org/88747 | 22:31 |
| fungi | okay, ^ proof things are indeed moving again | 22:31 |
| jesusaurus | fungi: it also moves bits of the ::jenkins::slave class into the ::openstack_projects module | 22:32 |
| jesusaurus | since most of that class is openstack-specific | 22:32 |
| fungi | yep, i see that | 22:32 |
| openstackgerrit | A change was merged to openstack-infra/config: Update zuul merger init script https://review.openstack.org/91211 | 22:33 |
| jesusaurus | and i also found that the term "bare slave" is overloaded, so i renamed the two things to simple_slave and thick_slave | 22:33 |
| fungi | the thin/thick terminology is good, i think | 22:33 |
| fungi | yes, my thinking exactly | 22:33 |
| openstackgerrit | A change was merged to openstack-infra/config: Add devstack job for Mistral project https://review.openstack.org/91335 | 22:34 |
| *** Ryan_Lane1 has joined #openstack-infra | 22:35 | |
| mgagne | at least we know https://review.openstack.org/#/c/89708/ didn't break XenAPI driver... | 22:36 |
| *** Ryan_Lane has quit IRC | 22:36 | |
| *** prad_ has quit IRC | 22:37 | |
| *** gyee has quit IRC | 22:39 | |
| jeblair | mgagne: yeah, i think they fixed that | 22:39 |
| *** doddstack has quit IRC | 22:40 | |
| *** weshay has quit IRC | 22:42 | |
| mattoliverau | . | 22:42 |
| zaro | jesusaurus: ++ on thick/thin | 22:42 |
| jesusaurus | :) | 22:42 |
| *** signed8bit has quit IRC | 22:43 | |
| *** signed8bit has joined #openstack-infra | 22:43 | |
| mgagne | jesusaurus: thanks for the cleanup =) | 22:43 |
| *** nati_ueno has quit IRC | 22:43 | |
| *** nati_ueno has joined #openstack-infra | 22:45 | |
| jesusaurus | de nada | 22:46 |
| *** UtahDave has quit IRC | 22:47 | |
| *** arnaud__ has joined #openstack-infra | 22:48 | |
| *** bknudson has joined #openstack-infra | 22:49 | |
| *** jhesketh_ has quit IRC | 22:51 | |
| *** andreykurilin_ has quit IRC | 22:52 | |
| openstackgerrit | Alex Conrad proposed a change to openstack-infra/jenkins-job-builder: support execute-on for postbuildscript https://review.openstack.org/90507 | 22:53 |
| *** jhesketh__ has joined #openstack-infra | 22:53 | |
| *** sweston has quit IRC | 22:53 | |
| *** bknudson has quit IRC | 22:54 | |
| *** jhesketh__ is now known as jhesketh | 22:54 | |
| jhesketh | Morning | 22:57 |
| *** david-lyle has quit IRC | 22:58 | |
| *** david-lyle has joined #openstack-infra | 22:58 | |
| *** david-lyle has quit IRC | 22:58 | |
| *** aysyd has quit IRC | 22:59 | |
| *** zns has quit IRC | 22:59 | |
| *** flaper87 is now known as flaper87|afk | 23:00 | |
| *** jamielennox|away is now known as jamielennox | 23:01 | |
| *** arnaud__ has quit IRC | 23:03 | |
| *** signed8bit has quit IRC | 23:09 | |
| *** bknudson has joined #openstack-infra | 23:09 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack-infra/gitdm: Add StevenK to the HP group list https://review.openstack.org/82706 | 23:09 |
| *** zns has joined #openstack-infra | 23:11 | |
| *** hemna is now known as hemna__ | 23:12 | |
| jeblair | phschwartz: something like 50-75% of dfw and iad nodes are failing to start now (i'm seeing lots of "ERROR" states in the logs) ord is great. | 23:16 |
| phschwartz | do you have instance id's for some that went to ERROR | 23:17 |
| phschwartz | I want to get an admin looking right now | 23:17 |
| jeblair | 1 sec | 23:17 |
| phschwartz | jeblair: ^^ | 23:17 |
| *** wenlock has quit IRC | 23:17 | |
| jeblair | phschwartz: 8d320eec-6f6a-416c-a526-a14437568029 at 2014-04-30 23:17:21,512 | 23:18 |
| jeblair | that was bare-precise-rax-dfw-3915000.slave.openstack.org | 23:18 |
| phschwartz | jeblair: let me pull up the nova log | 23:19 |
| *** luisg__ has joined #openstack-infra | 23:21 | |
| *** luisg_ has quit IRC | 23:21 | |
| *** mrodden has quit IRC | 23:31 | |
| *** sweston has joined #openstack-infra | 23:32 | |
| *** sweston has quit IRC | 23:38 | |
| harlowja | jeblair how do u currently exit gertty ?? :) | 23:38 |
| harlowja | kill `pid`? | 23:38 |
| anteaya | ctrl \ | 23:38 |
| jeblair | yeah ^, i need to add those daemon flags you pointed out. :) | 23:39 |
| *** sweston__ has joined #openstack-infra | 23:40 | |
| *** sweston__ is now known as sweston | 23:41 | |
| mattoliverau | anteaya: Do I need to generate a gerrit http password to use gertty? | 23:41 |
| anteaya | yes | 23:41 |
| anteaya | in settings you should see where | 23:42 |
| anteaya | click the generate button | 23:42 |
| anteaya | ta da | 23:42 |
| mattoliverau | thought so, just wanted to make sure :) | 23:45 |
| *** SumitNaiksatam has joined #openstack-infra | 23:45 | |
| *** mrodden has joined #openstack-infra | 23:45 | |
| *** nati_ueno has quit IRC | 23:48 | |
| *** nati_ueno has joined #openstack-infra | 23:48 | |
| anteaya | mattoliverau: sure | 23:48 |
| *** mrodden has quit IRC | 23:49 | |
| *** manishg has quit IRC | 23:54 | |
| *** nati_ueno has quit IRC | 23:54 | |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/config: JJB can take longer than 5 minutes to run https://review.openstack.org/91489 | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!