| fungi | davidlenwell: i'll have a look on the gerrit repo | 00:00 |
|---|---|---|
| davidlenwell | maybe something some place has the wrong date | 00:00 |
| fungi | the gerrit server itself has a sane system time | 00:01 |
| davidlenwell | the merge actually happens in the worker farm thing right ? | 00:02 |
| fungi | davidlenwell: yeah, merge commits look like they have the create commit date since they're created on gerrit | 00:02 |
| fungi | however changes which can merge without needing a merge commit keep their original commit date | 00:03 |
| davidlenwell | might actually be my systems fault | 00:03 |
| davidlenwell | I work in a vm most of the time .. and often suspend it for days at a time .. which seems to have screwed up its date | 00:04 |
| fungi | davidlenwell: it looks to me like your local system time is two weeks behind... do you still have a topic branch or other original commit around from when you were running git review? | 00:04 |
| lifeless | fungi: can you change max-servers to 40 please ? | 00:04 |
| davidlenwell | yep .. thats it .. | 00:04 |
| fungi | lifeless: can you submit a patch? i can manually change it, but puppet is going to change it back if i don't also stop puppet | 00:04 |
| lifeless | submitting patch too, of course | 00:04 |
| fungi | k | 00:04 |
| lifeless | we only have 40 virtualised test environments | 00:05 |
| lifeless | once we get more hardware back online ( Ng is pushing on that) we'll have more | 00:05 |
| fungi | i'm not 100% sure i get the context of those terms, though they are all words in my native tongue | 00:05 |
| clarkb | jogo: https://review.openstack.org/#/c/84690/1 that ready? | 00:05 |
| *** Ryan_Lane2 has quit IRC | 00:05 | |
| clarkb | zaro: sure let me look at it again | 00:05 |
| lifeless | fungi: each tripleo job uses a cluster of other VMs to run tests | 00:06 |
| *** andreaf2 has quit IRC | 00:06 | |
| lifeless | fungi: we call that cluster a 'testenv' | 00:06 |
| fungi | lifeless: okay, that. makes sense then | 00:06 |
| lifeless | fungi: we have a separate cluster of hosts that host those testenvs. | 00:06 |
| *** blamar has quit IRC | 00:06 | |
| lifeless | fungi: there are currently 40 such testenvs, but 55 jobs running would mean 15 jobs not being serviced | 00:06 |
| lifeless | fungi: during high churn periods thats fine - because slaves coming and going will contribute that 15 | 00:06 |
| fungi | lifeless: i've dropped it to 40 and very briefly stopped the puppet agent daemon | 00:07 |
| *** dcramer_ has joined #openstack-infra | 00:07 | |
| lifeless | fungi: but, during sustained overload, it means that we could end up with jobs with a slave active - so the countdown in jenkins starts - and then waiting too long to start doing anything, so they will timeout without cause | 00:07 |
| lifeless | fungi: which if that happened, would be self sustaining, so we'd basically not run anything to completion | 00:08 |
| *** talluri has joined #openstack-infra | 00:08 | |
| *** melwitt has quit IRC | 00:08 | |
| fungi | lifeless: yeah, makes sense without some sort of additional queuing layer | 00:08 |
| *** blamar has joined #openstack-infra | 00:09 | |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Drop max-servers to 40 for tripleo HP region. https://review.openstack.org/84903 | 00:10 |
| *** wenlock has quit IRC | 00:11 | |
| clarkb | zaro: approved | 00:12 |
| *** talluri has quit IRC | 00:12 | |
| openstackgerrit | A change was merged to openstack-infra/gerritlib: add getVersion and listPlugin commands and update replicate command https://review.openstack.org/69768 | 00:13 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Fix jshint linting errors https://review.openstack.org/84905 | 00:13 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Add in train tracks to status page https://review.openstack.org/84314 | 00:13 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Fix up filter issues introduced by jshint changes https://review.openstack.org/84112 | 00:13 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Tweak job layout css in status page https://review.openstack.org/84316 | 00:13 |
| *** mgagne has joined #openstack-infra | 00:13 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Adjust change headings on status page https://review.openstack.org/84317 | 00:13 |
| asselin | Hi, for nodepool, when it creates the image, a new keypair is created and used in the image. But when it comes time to ssh in an instance of the image, nodepool uses the ssh key provided in the nodepool.yaml file. Since they (obviously) don't match, nodepool is failing to ssh into the instance. | 00:14 |
| asselin | Am I missing some configuration somewhere? | 00:14 |
| *** SumitNaiksatam has quit IRC | 00:15 | |
| clarkb | asselin: yes, the setup script should install the public key for that key in the specified user's authorized_keys file | 00:15 |
| asselin | clarkb, sorry, I need to get this clear: The setup script running on the 'master' installs the new 'slave' public key in the 'master' user's authorized_keys file? | 00:17 |
| clarkb | asselin: no, the setup script provided in the nodepool.yaml. That script should when building the image install the authorized keys file for the username provided in nodepool.yaml | 00:20 |
| *** maru_afk is now known as marun | 00:20 | |
| fungi | clarkb: asselin: the "private-key" file path specified for each image in nodepool.yaml, right? | 00:20 |
| fungi | in http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb we use the local /home/nodepool/.ssh/id_rsa to log in and use puppet to install the public key component of that | 00:21 |
| *** yamahata has joined #openstack-infra | 00:22 | |
| *** gokrokve has quit IRC | 00:22 | |
| fungi | the $ssh_key we pass in through http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/slave.pp#n23 corresponds to it, right? (basically the same key the jenkins master authenticates to the slaves with)? | 00:22 |
| jogo | clarkb: I don't know wasn't aware of that patch | 00:23 |
| clarkb | jogo: are the existing python33 tests passing to a level you are comfy with? | 00:24 |
| clarkb | jogo: if so it is probably fine to start actually gating on it | 00:24 |
| jogo | clarkb: checking now | 00:24 |
| clarkb | jogo: awesome thanks | 00:24 |
| asselin | fungi, clarkb, I have private-key: /home/stack/.ssh/id_rsa | 00:24 |
| asselin | . For puppet, I need to configure it to put the corresponding public key in the instance? | 00:24 |
| fungi | asselin: so /home/stack/.ssh/id_rsa needs to exist on your nodepool server, first of all (that's telling it what its local ssh client should use to log into the new nodes to finalize their configuration) | 00:25 |
| fungi | asselin: and then, yes, the corresponding public part to that keypair needs to get installed (via puppet or whetever) during image creation into the ~/.ssh/authorized_keys file of the username nodepool will use to ssh into the nodes | 00:26 |
| *** blamar has quit IRC | 00:26 | |
| asselin | fungi, yes, that's on my nodepool server. I see lots of kesy in these files: modules/openstack_project/manifests/*.pp | 00:27 |
| jogo | clarkb: actually I am not | 00:27 |
| fungi | ours uses the jenkins account we create on the images, so in our cause it ends up in the /home/jenkins/.ssh/authorized_keys file on the nodepool template images | 00:27 |
| jogo | comnfortable with hacking gating on py33 yet | 00:27 |
| clarkb | jogo: ok, please leaev a comment (and -1) on the change | 00:27 |
| jogo | I think we first need to explore the impact of updating pep8 and flake8 versions | 00:27 |
| clarkb | jogo: hrm? | 00:28 |
| jogo | I don't think those will be an issue, but IMHO those should get precident | 00:28 |
| *** matsuhashi has joined #openstack-infra | 00:28 | |
| clarkb | jogo: oh yeah they totally work | 00:28 |
| clarkb | and if they don't we should burn them with fire | 00:28 |
| jogo | clarkb: I agree but, I would rather do this in the right order | 00:28 |
| clarkb | but agreed checking that first should be done (it is easy to do just push a WIP chagne that changes those numbers) | 00:28 |
| jogo | right | 00:29 |
| clarkb | the python33 job will run and tell you if it works | 00:29 |
| jogo | also py33 or py34? | 00:29 |
| fungi | jogo: py33 until we get trusty up in production | 00:30 |
| fungi | jogo: then we'll start adding py34 jobs and deprecating the py33 jobs | 00:30 |
| *** thuc has quit IRC | 00:30 | |
| *** prad has joined #openstack-infra | 00:30 | |
| *** prad has quit IRC | 00:30 | |
| fungi | in theory we should be able to run py27, py34 and pypy jobs on trusty workers | 00:30 |
| *** thuc has joined #openstack-infra | 00:31 | |
| jogo | fungi: ahh | 00:31 |
| fungi | which means an eventual end to specialized py3k-precise nodes once icehouse is eol support gets | 00:31 |
| * fungi no tyep good | 00:31 | |
| *** blamar has joined #openstack-infra | 00:32 | |
| asselin | fungi, so if I manually copy my stack user ssh key to the "ubuntu" user on the slave's authorized_keys it should work. "ubuntu" user is from the providers:images:username: entry in the nodepool.yaml file | 00:32 |
| fungi | i think we discussed that we might consider py33 for havana/icehouse scrapped and just transition to py34 on trusty for juno and drop the py3k-precise nodes sooner if possible | 00:32 |
| fungi | asselin: in theory yes, though that needs to ideally happen during image creation | 00:33 |
| *** german_ has quit IRC | 00:34 | |
| asselin | fungi, thanks....yes image creation is very slow (different issue I'd like to ask about) so I'd like to understand enough to test before automating. | 00:35 |
| *** thuc has quit IRC | 00:35 | |
| *** markmcclain has quit IRC | 00:36 | |
| asselin | fungi, clarkb thanks again for your help. I will continue tomorrow. | 00:36 |
| fungi | asselin: of course--we're always happy to see others use the tools we're developing, and understand they're not always trivial to implement | 00:39 |
| *** thuc has joined #openstack-infra | 00:39 | |
| *** rlandy has quit IRC | 00:40 | |
| *** thuc_ has joined #openstack-infra | 00:40 | |
| *** thuc_ has quit IRC | 00:41 | |
| *** thuc_ has joined #openstack-infra | 00:41 | |
| *** Adri2000 has quit IRC | 00:41 | |
| *** thuc__ has joined #openstack-infra | 00:43 | |
| *** thuc__ has quit IRC | 00:43 | |
| *** alexandra_ is now known as asettle | 00:43 | |
| *** thuc has quit IRC | 00:43 | |
| *** thuc has joined #openstack-infra | 00:44 | |
| *** thuc__ has joined #openstack-infra | 00:44 | |
| *** thuc__ has quit IRC | 00:45 | |
| *** thuc__ has joined #openstack-infra | 00:45 | |
| *** asettle is now known as alexsettle | 00:46 | |
| *** thuc_ has quit IRC | 00:46 | |
| *** alexsettle is now known as Alexandra | 00:46 | |
| *** thuc__ has quit IRC | 00:46 | |
| *** thuc_ has joined #openstack-infra | 00:47 | |
| lifeless | fungi: https://jenkins05.openstack.org/job/check-tripleo-ironic-undercloud-precise/153/console - that seems hung ? | 00:47 |
| *** dcramer_ has quit IRC | 00:47 | |
| fungi | agreed, nova list encountered "ERROR: HTTPConnectionPool(host='192.0.2.1', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 110] Connection timed out)" | 00:48 |
| *** mriedem has joined #openstack-infra | 00:48 | |
| *** thuc has quit IRC | 00:48 | |
| fungi | and the job is now doing nothing, hurtling toward a lengthy job timeout | 00:48 |
| fungi | lifeless: do you want the job aborted, or are you looking for troubleshooting assistance on it? | 00:49 |
| lifeless | fungi: I'm wondering if something is systemically wrong | 00:50 |
| lifeless | fungi: e.g. like a dropped tcp connection | 00:50 |
| lifeless | from slave to jenkins | 00:50 |
| fungi | lifeless: doesn't seem so... jenkins appears to run keepalives to slaves and notices when they go away/stop responding fairly quickly | 00:51 |
| lifeless | ok | 00:51 |
| lifeless | problem in test script then, will ignore | 00:51 |
| fungi | lifeless: i can ssh into the slave and see if it's still doing stuff, or place the node on hold and add an ssh key for you | 00:52 |
| *** julim has quit IRC | 00:55 | |
| *** julim has joined #openstack-infra | 00:56 | |
| fungi | or i can go have an evening. yes, i think i might just do that | 00:56 |
| *** thuc_ has quit IRC | 00:57 | |
| *** julim has quit IRC | 01:00 | |
| *** harlowja is now known as harlowja_away | 01:01 | |
| *** dcramer_ has joined #openstack-infra | 01:01 | |
| lifeless | fungi: is possible to change the place in the for a job ? | 01:02 |
| lifeless | fungi: oy I just missed you :P | 01:03 |
| lifeless | clarkb: ^ | 01:03 |
| *** yaguang has joined #openstack-infra | 01:03 | |
| portante | fungi, clarkb, lifeless: can somebody point me at the official license header template for python source? | 01:03 |
| fungi | lifeless: still here, but not succeeding in parsing your question | 01:03 |
| portante | I was using: http://docs.openstack.org/developer/hacking/ | 01:04 |
| openstackgerrit | Joe Gordon proposed a change to openstack-dev/hacking: Test reving the dependencies https://review.openstack.org/84913 | 01:04 |
| lifeless | fungi: I have a job - 84361 which may fix the timeouts on all the ironic-undercloud jobs if landed | 01:04 |
| lifeless | fungi: but it hasn't actually run and its deep in the queue | 01:04 |
| lifeless | fungi: I'm wondering if its possible to promote it to the front | 01:04 |
| fungi | lifeless: i can enqueue it in the gate if it's still languishing in check waiting for resources, yes | 01:05 |
| *** matsuhashi has quit IRC | 01:05 | |
| lifeless | fungi: I mean front of check queue | 01:05 |
| lifeless | fungi: erm check-tripleo | 01:06 |
| * portante ... **crickets** ... | 01:06 | |
| lifeless | portante: is there one? | 01:06 |
| *** matsuhashi has joined #openstack-infra | 01:06 | |
| portante | I am looking to see if there is an official template somewhere, or if what is on that hacking page above is the one to use | 01:06 |
| portante | lifeless: | 01:06 |
| fungi | lifeless: i can try to reorder that, though i've only ever used zuul promote on the gate pipeline, so no guarantees | 01:07 |
| *** julim has joined #openstack-infra | 01:07 | |
| lifeless | fungi: please, that would be super useful | 01:07 |
| fungi | lifeless: seems to have worked, though using zuul promote on an independent queue still seems to cancen and resubmit all jobs | 01:09 |
| fungi | s/cancen/cancel/ | 01:09 |
| lifeless | fungi: it has to I think, because jenkins | 01:09 |
| fungi | anyway, i'm really not here now ;) | 01:09 |
| jogo | clarkb: commented | 01:10 |
| *** yamahata has quit IRC | 01:11 | |
| *** mrodden has quit IRC | 01:11 | |
| *** saschpe has quit IRC | 01:11 | |
| *** julim has quit IRC | 01:12 | |
| fungi | portante: i think the closest you're going to get it http://git.openstack.org/cgit/openstack-dev/hacking/tree/hacking/core.py#n142 | 01:12 |
| *** bhuvan_ has quit IRC | 01:13 | |
| portante | fungi: thanks | 01:13 |
| fungi | (that's the text _check_for_exact_apache() compares against) | 01:13 |
| portante | so we don't need any "Copyright" strings in there anymore? | 01:13 |
| portante | huh | 01:13 |
| portante | k | 01:13 |
| fungi | portante: it doesn't check the copyright notices because they're not part of the license and irregular in nature | 01:13 |
| portante | ah | 01:13 |
| fungi | portante: copyright notices are up to personal/project discretion. i think projects benefit from adding and maintaining them, but that's just my opinion | 01:14 |
| portante | fungi: okay, thanks | 01:15 |
| portante | notmyname: ^^^ | 01:15 |
| *** dcramer_ has quit IRC | 01:15 | |
| portante | notmyname: your call, I can add an OpenStack Copyright if you'd like | 01:15 |
| notmyname | portante: not important to me personally, and IIRC, the foundation doesn't want to be assigned copyright | 01:16 |
| fungi | portante: copyrighting something OpenStack is fairly inaccurate ;) the author(s) of the work or their employers in a work-for-hire scenario is/are the copyright holder(s) | 01:16 |
| portante | okay | 01:17 |
| portante | thanks guys | 01:17 |
| jeblair | (unless you're moving around code that is already under openstack foundation copyright as opposed to writing new code) | 01:17 |
| fungi | at the moment, the only new works which should technically be getting a "copyright openstack foundation" are those works written by the handful of us who are directly employed by the foundation, or working on a contract for it | 01:17 |
| fungi | yes | 01:17 |
| jeblair | portante: there are a lot of openstack foundation copyright headers around because at the start of the project, rackspace assigned all of its openstack-related copyrights to the foundation (by way of the llc) | 01:18 |
| notmyname | "Copyright OpenStack, because jeblair told me this had to be in Swift" | 01:18 |
| fungi | preserving existing copyright counts, for code copying, agreed | 01:18 |
| fungi | heh | 01:18 |
| *** saschpe has joined #openstack-infra | 01:18 | |
| jeblair | notmyname: ;) | 01:19 |
| * fungi is apparently actually really bad at not being here | 01:19 | |
| jeblair | fungi: you're the worst at it :) | 01:20 |
| * jeblair goes away | 01:20 | |
| portante | jeblair: thanks | 01:23 |
| * portante knows you read that! ;) | 01:23 | |
| *** nosnos has joined #openstack-infra | 01:29 | |
| *** rfolco has quit IRC | 01:32 | |
| *** VijayTripathi has quit IRC | 01:35 | |
| *** julim has joined #openstack-infra | 01:37 | |
| *** Adri2000 has joined #openstack-infra | 01:42 | |
| *** Alexandra is now known as alex-away | 01:46 | |
| *** sahumada has quit IRC | 01:53 | |
| *** sahumada has joined #openstack-infra | 01:57 | |
| *** wenlock has joined #openstack-infra | 02:00 | |
| *** zns has quit IRC | 02:01 | |
| *** alff has quit IRC | 02:08 | |
| *** alff_ has quit IRC | 02:08 | |
| *** thuc has joined #openstack-infra | 02:08 | |
| *** HenryG has quit IRC | 02:08 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Move status stylesheets into css file https://review.openstack.org/84921 | 02:09 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Add in sparklines to status page pipelines https://review.openstack.org/84922 | 02:09 |
| *** talluri has joined #openstack-infra | 02:10 | |
| *** alff_ has joined #openstack-infra | 02:12 | |
| *** alff has joined #openstack-infra | 02:12 | |
| *** alff has quit IRC | 02:12 | |
| *** alff_ has quit IRC | 02:12 | |
| *** thuc has quit IRC | 02:13 | |
| *** talluri has quit IRC | 02:14 | |
| *** mgagne1 has joined #openstack-infra | 02:16 | |
| *** mgagne has quit IRC | 02:18 | |
| mordred | jhesketh: why move the js stuff down a level? https://review.openstack.org/#/c/84069/ ? | 02:21 |
| mordred | jhesketh: if the entry isn't in the main tox file, it'll kinda be hard to re-use the gate tox infrastructure we already have ... | 02:21 |
| *** jlk has left #openstack-infra | 02:22 | |
| jhesketh | mordred: well I figured we could work around the gate with some JJB shell | 02:22 |
| jhesketh | but I moved it so that if we ever separate it out entirely from zuul it's ready to go | 02:23 |
| jhesketh | and it kind of litters the repository | 02:23 |
| jhesketh | mordred: did you read the scrollback yesterday when we spoke about the javascript changes? | 02:23 |
| *** Alexandra has joined #openstack-infra | 02:23 | |
| mordred | no. let me go find that | 02:23 |
| mordred | cana you tl;dr me? | 02:23 |
| jhesketh | basically we're not sure if we want to introduce all the extra jshint stuff + complexity | 02:24 |
| *** Alexandra has left #openstack-infra | 02:24 | |
| jhesketh | (you were pinged on it yesterday morning my time, so around 2200-2300 UTC) | 02:24 |
| *** Alexandra has joined #openstack-infra | 02:26 | |
| mordred | gotcha | 02:26 |
| mordred | so - if we weren't re-inventing grunt and bower in shell script and vendoring in things | 02:26 |
| mordred | I'd say "yeah, we shoudln't add the tools" | 02:26 |
| mordred | but we're adding tools - just custom written ones | 02:26 |
| mordred | so in my opinion, now that it's gotten complex, we should start using the tools taht are designed fo rthe job at hand | 02:27 |
| mordred | that said - I understand reluctance | 02:27 |
| *** Alexandra is now known as asettle | 02:28 | |
| *** asettle is now known as Alexandra | 02:29 | |
| *** nkinder has quit IRC | 02:29 | |
| clarkb | mordred: its less about tools and moer the way they were introduced | 02:32 |
| clarkb | mordred: that chagne is massive with little to no documentation or comments | 02:32 |
| clarkb | mordred: also it should probably be about 3 changes | 02:32 |
| clarkb | one to add framework with maybe linting, another to add unit tests and another to add tools | 02:32 |
| clarkb | as is its just hard to understand | 02:32 |
| *** MarkAtwood has joined #openstack-infra | 02:34 | |
| *** david-lyle has joined #openstack-infra | 02:34 | |
| mordred | clarkb: ok. I can do that | 02:34 |
| clarkb | mordred: the other concern was less css | 02:35 |
| clarkb | because packaging | 02:35 |
| clarkb | and are the js tools really so insane that we need 10 config files to do that | 02:35 |
| clarkb | python needs 2 | 02:35 |
| mordred | meh | 02:35 |
| mordred | it's how js works | 02:35 |
| clarkb | mordred: its not meh though | 02:35 |
| mordred | it is | 02:35 |
| clarkb | it is incredibly confusing | 02:35 |
| mordred | it's not really | 02:35 |
| lifeless | mordred: oh hai | 02:36 |
| mordred | and we're alraedy using it | 02:36 |
| mordred | in another infra project | 02:36 |
| clarkb | mordred: zuul isn't | 02:36 |
| clarkb | zuul has a different core group I am going to be a bit more picky | 02:36 |
| clarkb | because 10 config files is insane | 02:36 |
| clarkb | mordred: it seems like there was a lot of overlap in them | 02:36 |
| mordred | ok. that's fine. anyway - I'll rework the changes to be more readable | 02:36 |
| clarkb | installjs vs bootstrap | 02:36 |
| clarkb | and dependencies listed in multiple places and so on | 02:36 |
| mordred | some of them were copies that are not needed | 02:36 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1301027 https://review.openstack.org/84558 | 02:37 |
| uvirtbot | Launchpad bug 1301027 in neutron/havana "Permission denied error in unit tests" [High,Fix committed] https://launchpad.net/bugs/1301027 | 02:37 |
| mordred | so, like I said, I'll rework the patches based on that feedback | 02:37 |
| clarkb | ok thanks | 02:37 |
| *** esker has joined #openstack-infra | 02:37 | |
| *** mriedem has quit IRC | 02:37 | |
| *** mrodden has joined #openstack-infra | 02:39 | |
| clarkb | mordred: what does bower do? grunt is basically make and karma is the test suite | 02:40 |
| mordred | yes | 02:40 |
| mordred | bower is the thign that does what "etc/status/fetch-dependencies.sh" does | 02:40 |
| mordred | it fetches web-style dependencies - that is, things that nobody ever intended would _install_ as a library and be consumed | 02:41 |
| *** Alexandra_ has joined #openstack-infra | 02:41 | |
| mordred | but instead are designed to be included into a website | 02:41 |
| *** Alexandra_ has quit IRC | 02:41 | |
| *** asettle has joined #openstack-infra | 02:41 | |
| mordred | it pulls them into bower_components, and then the build chain knows where to find them, so that when it generates the tarball, all is minified properly | 02:41 |
| *** nkinder has joined #openstack-infra | 02:42 | |
| clarkb | so python setup.py sdist equivalent? | 02:42 |
| mordred | not entirely | 02:42 |
| mordred | it's more like pip - but for a different type of thing | 02:42 |
| jhesketh | mordred: okay, if you're going to rework it again would you mind doing it at the new tail of my changes? | 02:43 |
| mordred | so, like, you use it to grab bootstrap and put it into a local dir - so that you don't have to vendor bootstrap into your tree | 02:43 |
| mordred | jhesketh: absolutely | 02:43 |
| *** Alexandra has quit IRC | 02:43 | |
| jhesketh | mordred: also I'd like to see this stuff inside the status folder to keep zuul cleaner | 02:43 |
| jhesketh | it does appear to add a lot of config junk | 02:43 |
| clarkb | mordred: it seemed like we define deps in bower separate of grunt | 02:44 |
| mordred | that's right | 02:44 |
| mordred | grunt does not do depends | 02:44 |
| clarkb | is that right? that seems wrong from a we tested X and package Y perspective | 02:44 |
| mordred | it doesn't assume a world where there would bea distro package of bootstrap that would be used | 02:45 |
| mordred | because that world doesn't raelly exit | 02:45 |
| mordred | exist | 02:45 |
| mordred | you declare the version of bootstrap you want, and bower pulls that down, and then when grunt runs, it uses what's now in the tree | 02:45 |
| mordred | it's a bit like vendoring without having to check the vendored code into the repo | 02:45 |
| clarkb | mordred: oh so grunt does the bower step? | 02:45 |
| mordred | no | 02:45 |
| clarkb | how do we make sure what we package is what we test then? | 02:46 |
| mordred | that's in https://review.openstack.org/#/c/82307/4/tools/install-js.sh | 02:46 |
| mordred | it is - it's just a step before grunt | 02:46 |
| mordred | so - you run npm install to install any npm depends | 02:49 |
| mordred | in our case, those are all dev/test depends and not things that get installed into the site eventually | 02:49 |
| clarkb | which is karma and test depends or jshint | 02:49 |
| clarkb | right | 02:49 |
| mordred | right | 02:49 |
| mordred | then, using bower, you install the libraries that actually _do_ get installed into the site | 02:49 |
| *** julim has quit IRC | 02:50 | |
| mordred | clarkb: so then, https://review.openstack.org/#/c/82307/4/status/src/index.html | 02:51 |
| mordred | line 24 | 02:51 |
| mordred | that chunk is referencing files in the bower_components dir | 02:51 |
| clarkb | but it isn't a script so it doesnt' do things at run time right? | 02:52 |
| mordred | and the minifier will combine them into /js/libs.js and will replace that section with the one that references /js/libs.js instead | 02:52 |
| clarkb | it is build time? | 02:52 |
| clarkb | ah gotcha | 02:52 |
| clarkb | so you end up with a massive file with minified stuff | 02:52 |
| *** thuc has joined #openstack-infra | 02:52 | |
| mordred | yup | 02:52 |
| mordred | and you know for certain that what you deploy is what you tested :) | 02:53 |
| *** thuc_ has joined #openstack-infra | 02:53 | |
| *** weshay has joined #openstack-infra | 02:53 | |
| *** pcrews_ has quit IRC | 02:54 | |
| *** fifieldt_ is now known as fifieldt | 02:54 | |
| mordred | but honestly - great point in terms of code drop - I put the unittest stuff in there to show that we can do it and where it goes - but since we don't actually HAVE that | 02:54 |
| mordred | just not having it in tree as step one will be hella clearer :) | 02:55 |
| *** thuc has quit IRC | 02:57 | |
| *** mrodden has quit IRC | 03:00 | |
| *** mrodden has joined #openstack-infra | 03:02 | |
| *** matsuhashi has quit IRC | 03:05 | |
| *** thuc_ has quit IRC | 03:08 | |
| *** thuc has joined #openstack-infra | 03:08 | |
| *** yjiang has joined #openstack-infra | 03:11 | |
| *** thuc has quit IRC | 03:11 | |
| *** talluri has joined #openstack-infra | 03:11 | |
| clarkb | fungi: lifeless: so are we still leaking floating ips? it wasn't clear to me if we were still experiencing that | 03:12 |
| *** esker has quit IRC | 03:12 | |
| *** esker has joined #openstack-infra | 03:13 | |
| clarkb | there appaer to be a relatively largish number of tests running now compared to yesterday | 03:13 |
| *** sdake has quit IRC | 03:14 | |
| *** talluri has quit IRC | 03:16 | |
| lifeless | clarkb: I'll stop my workaround and we'll find out | 03:19 |
| *** mrodden has quit IRC | 03:22 | |
| *** nosnos has quit IRC | 03:23 | |
| clarkb | lifeless: and if we are, I will whip up a patch that checks fixed ip against the server's fixed ip | 03:27 |
| *** gokrokve has joined #openstack-infra | 03:36 | |
| *** gokrokve has quit IRC | 03:36 | |
| *** basha_ has joined #openstack-infra | 03:36 | |
| *** gokrokve has joined #openstack-infra | 03:37 | |
| *** VijayTripathi has joined #openstack-infra | 03:38 | |
| *** thuc has joined #openstack-infra | 03:39 | |
| zaro | fungi: you are correct getVersion is unused. it's a nice to have. i probably should have added that method in a seperate change. | 03:39 |
| *** maxbit has quit IRC | 03:56 | |
| *** basha_ has quit IRC | 03:56 | |
| *** harlowja_away is now known as harlowja | 04:00 | |
| *** SumitNaiksatam has joined #openstack-infra | 04:02 | |
| lifeless | clarkb: its looking ok so far, but we really need a few hours - cycled nodes - to be sure | 04:03 |
| *** thuc has quit IRC | 04:03 | |
| clarkb | rgr | 04:03 |
| lifeless | clarkb: I'll check before I sleep and if leaking put the workaround back | 04:03 |
| *** thuc has joined #openstack-infra | 04:04 | |
| *** thuc has quit IRC | 04:07 | |
| lifeless | pleia2: I see there is a fedora job on check-tripleo now | 04:08 |
| lifeless | pleia2: but no images scheduled or live | 04:08 |
| lifeless | pleia2: so this is making all runs fail with NOT REGISTERED | 04:09 |
| lifeless | pleia2: how do you feel about us backing that out ? Or is fedora veryveryclose ? | 04:09 |
| pleia2 | lifeless: hmm, there were some earlier, did any of them work? | 04:09 |
| *** nosnos has joined #openstack-infra | 04:09 | |
| clarkb | we definitely had an image | 04:09 |
| pleia2 | the slaves seemed to be building fine earlier | 04:09 |
| pleia2 | it should work, but we might want to regroup with derekh tomorrow since he'll know more about actually running the tests I think | 04:10 |
| *** thuc has joined #openstack-infra | 04:11 | |
| lifeless | theres a slave limit of 1 for fedora | 04:12 |
| lifeless | AIUI we need a limit => jenkinses | 04:13 |
| lifeless | or the other jenkins will barf, no ? | 04:13 |
| lifeless | actually | 04:13 |
| lifeless | I know what I think we should do | 04:13 |
| lifeless | make the fedora job nonvoting | 04:13 |
| lifeless | as its new | 04:13 |
| lifeless | nonvoting in a nonvoting queue :) | 04:13 |
| pleia2 | ah yes, I think he put it is as 1 because it's new, but you're right, it really should be non-voting | 04:14 |
| clarkb | 1 should be fine it will scale up | 04:14 |
| *** matsuhashi has joined #openstack-infra | 04:15 | |
| lifeless | oh right, min-ready | 04:15 |
| clarkb | maybe the jenkins job label doesnt match slave label? | 04:15 |
| pleia2 | check-tripleo-overcloud-f20 | 04:15 |
| pleia2 | check-tripleo-overcloud-f20 | 04:16 |
| pleia2 | they're the same | 04:16 |
| lifeless | nonvoting is enough to make me happy | 04:16 |
| *** VijayTripathi has quit IRC | 04:16 | |
| clarkb | image build must've broken then? /me looks at web hosted logs | 04:17 |
| clarkb | pleia2: the label is tripleo-f20 according to nodepool | 04:19 |
| pleia2 | clarkb: oh, the name of the slave | 04:19 |
| pleia2 | right, you said that, let me check the patches again | 04:20 |
| pleia2 | node: 'tripleo-f20' | 04:20 |
| clarkb | ya that looks correct | 04:21 |
| clarkb | I wonder if we have simply been unable to build a new image beacuse all of the quota has been used to kill that backlog | 04:22 |
| clarkb | I don't see tripleo-f20 showing up in the image logs the last few days | 04:22 |
| *** dcramer_ has joined #openstack-infra | 04:22 | |
| *** mrodden has joined #openstack-infra | 04:24 | |
| lifeless | clarkb: quota is 60 | 04:24 |
| *** BadCub01 has quit IRC | 04:28 | |
| *** michchap has quit IRC | 04:29 | |
| *** michchap has joined #openstack-infra | 04:29 | |
| *** masayukig has quit IRC | 04:32 | |
| *** masayukig has joined #openstack-infra | 04:32 | |
| *** talluri has joined #openstack-infra | 04:34 | |
| *** talluri has quit IRC | 04:34 | |
| *** talluri has joined #openstack-infra | 04:34 | |
| *** ArxCruz has quit IRC | 04:36 | |
| *** mihgen has joined #openstack-infra | 04:37 | |
| *** ArxCruz has joined #openstack-infra | 04:38 | |
| *** matsuhashi has quit IRC | 04:40 | |
| *** weshay has quit IRC | 04:43 | |
| mordred | dstufft: tchaypo has made a patch for pip that makes us very happy | 04:44 |
| *** resker has joined #openstack-infra | 04:44 | |
| *** resker has quit IRC | 04:44 | |
| dstufft | which is that | 04:45 |
| mordred | dstufft: https://github.com/jamezpolley/pip/commit/66dff9a8a3bf95d105792f2aae3dd7caa65b5253 | 04:45 |
| mordred | dstufft: it fixxes the issue that file:// and http:// urls exhibit different behvior | 04:46 |
| *** matsuhashi has joined #openstack-infra | 04:46 | |
| mordred | which has been plaguing me as a person | 04:46 |
| dstufft | sigh | 04:46 |
| dstufft | fucking file:// urls | 04:46 |
| dstufft | buggiest things ever | 04:47 |
| mordred | yah | 04:47 |
| mordred | BUT | 04:47 |
| mordred | really helpful in some cases | 04:47 |
| dstufft | yea yea | 04:47 |
| *** esker has quit IRC | 04:47 | |
| mordred | (I promise, there really are cases - I'm not lying) | 04:47 |
| mordred | (ok, I'm lying, but not MUCH) | 04:47 |
| dstufft | ;) | 04:47 |
| dstufft | oh | 04:48 |
| dstufft | all our tests are broken right now because of wheel | 04:48 |
| dstufft | lol | 04:48 |
| mordred | NICE | 04:48 |
| dstufft | I forgot about that | 04:48 |
| dstufft | also apparently someone merged or commited pep8 violations | 04:49 |
| tchaypo | dstufft: i hear you're a pip maintainer type | 04:50 |
| mordred | dstufft: if you want gating - we've got some you can use ... | 04:50 |
| clarkb | lifeless: does the base fedora image existing in your cloud after everything got blown away? | 04:50 |
| clarkb | running down ideas for why the image doesn't show up in the image log at all | 04:50 |
| clarkb | and not being able to boot the base image may explain it | 04:50 |
| dstufft | mordred: we theortically test all PRs before they land, probably someone commited directly to develop, or they merged a PR and ignored the failures because everything else is red now anyways so who wants to read the failures to make sure a change isn't introducing more failures | 04:51 |
| dstufft | tchaypo: yea | 04:51 |
| clarkb | lifeless: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb#n404 | 04:51 |
| tchaypo | dstufft:My patch so far is based on bashing out a nasty hack and being surprised that it works. I haven't yet looked at what kind of coding standards or testing pip wants before it can accept the patch | 04:51 |
| *** skraynev_afk is now known as skraynev | 04:52 | |
| mordred | dstufft: yah. I'm just trolling because $wine | 04:52 |
| tchaypo | mordred: but it | 04:52 |
| tchaypo | s only 4pm! | 04:52 |
| mordred | tchaypo: that's four hours after wine drinking starts! | 04:52 |
| dstufft | tchaypo: your patch looks reasonable, it'd be nice to get some tests for it, but like all our tests are borken right now because wheel | 04:53 |
| tchaypo | baaha | 04:53 |
| * mordred goes to do more not-IRC | 04:53 | |
| tchaypo | it's nice to know that we aren't the only project broken because wheel | 04:53 |
| mordred | which may involve kevin spacey | 04:53 |
| clarkb | mordred: house of cards? | 04:53 |
| mordred | clarkb: yah. we're still on season one | 04:53 |
| dstufft | someday I'm going to quit all this computering and become a goat herder | 04:53 |
| mordred | dstufft: +1000 | 04:53 |
| lifeless | clarkb: I created one yes | 04:53 |
| clarkb | mordred: I will not spoil anything then | 04:53 |
| lifeless | clarkb: you can check with glance list-images | 04:53 |
| mordred | dstufft: I will fund your herd if I can help | 04:53 |
| lifeless | or image-list or whatever it is | 04:54 |
| clarkb | lifeless: ya nova image list I can check tomorrow | 04:54 |
| dstufft | http://d.stufft.io/image/0C3S140L3d0Q apparently github puts build stats on the overbiew now, and lol our PRs | 04:54 |
| lifeless | clarkb: if you like proxying through apis sure | 04:54 |
| clarkb | lifeless: I kinda do, then I only need one client (when can openstaclient be a thing?) | 04:54 |
| lifeless | clarkb: nova image list will go away eventually | 04:54 |
| clarkb | lifeless: hopefully not before openstack iamge list is a thing | 04:55 |
| mordred | clarkb: openstackclient actually works now | 04:55 |
| mordred | clarkb: I've used it to do things | 04:55 |
| clarkb | oh? I should install it then | 04:55 |
| mordred | clarkb: I keep meaning to learn to use it for more things | 04:55 |
| mordred | it'snot 100% complete - but I think it has the things we're likely to use | 04:55 |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/reviewstats: Update solum core-team https://review.openstack.org/84939 | 04:57 |
| dstufft | mordred: your not IRC looks vaguely similar to my not IRC | 04:57 |
| mordred | dstufft: not-IRC is harder to do than IRC - there are 4 more characters | 04:57 |
| clarkb | dstufft: do I need to not spoil things for you too? | 04:58 |
| dstufft | clarkb: oh I meant that I seem to awlays end up back on IRC when I say I'm going to go do something else ;) | 04:58 |
| dstufft | I'm not watching that house of cards show | 04:58 |
| mordred | clarkb: don't say things in scrollback | 04:59 |
| mordred | I do, you know, sometimes read what happened during not-IRC | 04:59 |
| clarkb | don't worry I will be good | 05:00 |
| clarkb | dstufft: its actually quite good | 05:00 |
| * mordred throws wet cats into the air | 05:00 | |
| dstufft | mordred: dumbledore kills snape | 05:00 |
| mordred | dstufft: you are a bad bad boy | 05:00 |
| clarkb | I can do Game of Thrones spoilers too but I won't | 05:01 |
| dstufft | I need to catch up on both the books and the show :/ | 05:01 |
| clarkb | dstufft: reading books is how I do actual no irc and make sure it doesn't become irc | 05:01 |
| clarkb | currently reading green mars, I should go do that soon and fall asleep | 05:01 |
| *** thuc has quit IRC | 05:01 | |
| mordred | I was about to do not-IRC, but then sandy started spraying medicine on the cat | 05:02 |
| dstufft | somehow when the choice is "enjoy myself" and "get mad at computers" I pick the latter far more than is healthy | 05:02 |
| mordred | so I went back to IRC for a minute | 05:02 |
| *** yamahata has joined #openstack-infra | 05:02 | |
| mordred | dstufft: computers are extremely vexing | 05:02 |
| clarkb | but for some reason we keep smashing our keyboards | 05:02 |
| *** thuc has joined #openstack-infra | 05:02 | |
| clarkb | also this software stuff that people write is pretty terrible | 05:02 |
| nibalizer | IRC is my default animation when idle | 05:02 |
| nibalizer | that and typing 'arst' + ^U over and over again | 05:03 |
| StevenK | mordred: What's did the cat do to deserve that? :-P | 05:03 |
| *** jcoufal has joined #openstack-infra | 05:03 | |
| mordred | StevenK: countless countless things | 05:03 |
| dstufft | tchaypo: I think I just merged stuff to make develop pass tests again for pip | 05:03 |
| dstufft | tchaypo: so you probably want to rebase or something | 05:03 |
| dstufft | but you probably also don't want to run the tests locally | 05:04 |
| dstufft | unless you have nothing to do for like 3 hours | 05:04 |
| StevenK | So pip takes after Launchpad? | 05:04 |
| dstufft | at least it doesn't use up like 15GB of temp space anymore | 05:04 |
| StevenK | (Without using a massive machine and 20 LXC containers, Launchpad's test suite takes about 6 hours) | 05:04 |
| clarkb | dstufft: uh what? | 05:05 |
| clarkb | dstufft: does it build packages for everythign on pypi? | 05:05 |
| clarkb | and install them? | 05:05 |
| *** thuc has quit IRC | 05:06 | |
| dstufft | StevenK: it takes about 20 minutes per CPython (2.6, 2.7, 3.2, 3.3, and 3.4) and then 45-50 minutes for PyPy | 05:06 |
| dstufft | clarkb: we have a directory full of test data that we copy into a unique per test case temporary directory | 05:06 |
| dstufft | repeat that 400 some times | 05:06 |
| dstufft | also in that test data is 400 some virtualenvs | 05:06 |
| *** LinuxJedi has quit IRC | 05:07 | |
| *** mrmartin has joined #openstack-infra | 05:07 | |
| StevenK | mordred: Challenge accepted, re: -dev mail | 05:07 |
| clarkb | dstufft: wow gg | 05:07 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Accept file objects as well as filenames https://review.openstack.org/84942 | 05:08 |
| clarkb | StevenK: yes we should have the summit for M there | 05:08 |
| StevenK | Haha | 05:08 |
| dstufft | clarkb: we didn't used to copy, but using a file:// url (especially an editable one) mutates the directory poitned to by the file:// url | 05:08 |
| dstufft | so we had lots of non deterministic tests that would fail depending on what order they ran in | 05:09 |
| dstufft | also for awhile we had pytest-xdist running to speed things up, which was lol when trying to run two tests at the same time that both wanted to mutate the same directory | 05:09 |
| clarkb | StevenK: I approve of M summit in sydney | 05:09 |
| *** ArxCruz has quit IRC | 05:09 | |
| clarkb | StevenK: now we just have to make it so | 05:09 |
| clarkb | dstufft: nice | 05:09 |
| tchaypo | How far away is M? | 05:10 |
| clarkb | dstufft: /me knows how that goes having made things work with testr | 05:10 |
| tchaypo | because at the moment our convention center is a tent on a barge | 05:10 |
| clarkb | tchaypo: we are doing J in atlanta | 05:10 |
| dstufft | we don't have pytest-xdist anymore because it thought it was cool to randomly error during collection | 05:10 |
| tchaypo | and our exhibition center is a tent that you can only access by taking a train to central and then transferring to a bus | 05:10 |
| dstufft | :/ | 05:10 |
| clarkb | K - paris, L - wherever, M - Sydney | 05:10 |
| tchaypo | so M is november next year? | 05:11 |
| tchaypo | not going to work | 05:11 |
| tchaypo | Try 2016, we might have facilities by then | 05:11 |
| clarkb | tchaypo: don't crush my dreams | 05:11 |
| clarkb | tchaypo: maybe we can get away with a hotel conference center and just do the design summit in sydney | 05:11 |
| clarkb | the conference can go to a big conference center | 05:11 |
| *** yamahata has quit IRC | 05:12 | |
| tchaypo | I still think we should get RH to host the mid-cycle in Brisbane in the week after pycon-au | 05:12 |
| tchaypo | complaints from seattleites aside | 05:12 |
| *** alff has joined #openstack-infra | 05:12 | |
| *** yamahata has joined #openstack-infra | 05:12 | |
| *** alff_ has joined #openstack-infra | 05:12 | |
| *** VijayTripathi has joined #openstack-infra | 05:12 | |
| clarkb | tchaypo: Brisbane sounds infinitely better than sunnyvale | 05:12 |
| jhesketh | hypothetically, if pycon-au wasn't going to have an openstack miniconf (but still have openstack content in the main conference), would that affect a mid-cycle plans? | 05:16 |
| tchaypo | jhesketh: my understanding is that there will be a mid-cycle; the timing and venue will be determined by who wants to host | 05:17 |
| tchaypo | I'm suggesting brisbane after pycon because it would be nice for me, but we'd have to find someone who can host 40+ for a week (assuming attendance is similar to sunnyvale) | 05:18 |
| jhesketh | well there's a good chance we can get the pycon-au organisers to help | 05:18 |
| jhesketh | or somebody to sponsor | 05:18 |
| tchaypo | to be clear, this is tripleo we're talking about, not openstack generally | 05:19 |
| jhesketh | ah okay | 05:19 |
| clarkb | tchaypo: that said, the seattle office is much changed | 05:20 |
| clarkb | tchaypo: and has room for people | 05:20 |
| tchaypo | does it have guest wifi? | 05:21 |
| clarkb | yup | 05:21 |
| tchaypo | that's unexpected | 05:21 |
| tchaypo | can we roll that out to all the other offices? | 05:21 |
| clarkb | we'e even got the super secret network | 05:21 |
| clarkb | but apparently we can't use that for guests or something | 05:21 |
| *** sdake has joined #openstack-infra | 05:23 | |
| *** mrmartin has quit IRC | 05:24 | |
| clarkb | you know I wonder if we could get them to make a temporary secret network for things like that | 05:24 |
| clarkb | setup a second ssid on a different vlan and pipe that through the same NAT | 05:24 |
| clarkb | or just make it a permanent secret network with weekly passphrase changes | 05:24 |
| *** wenlock_ has joined #openstack-infra | 05:26 | |
| lifeless | super secret network ? | 05:26 |
| clarkb | melody | 05:27 |
| *** asettle has quit IRC | 05:28 | |
| *** alff_ has quit IRC | 05:31 | |
| *** alff has quit IRC | 05:31 | |
| *** harlowja is now known as harlowja_away | 05:32 | |
| jhesketh | fungi: ping | 05:37 |
| clarkb | jhesketh: it is 1:30am EDT for fungi right now | 05:38 |
| jhesketh | yeah, wasn't necessarily expecting a response :p | 05:38 |
| mattoliverau | If there is a mid-cycle in Brisbane, we should move it to the goldcoast, rent out wet and wild, and then everybody will want to come :) Water slides, sun, and openstack, what could be better ;) | 05:45 |
| *** LinuxJedi has joined #openstack-infra | 05:49 | |
| *** CaptTofu has joined #openstack-infra | 05:52 | |
| mattoliverau | I'm off, cya all tomorrow. | 05:56 |
| *** andreaf2 has joined #openstack-infra | 05:57 | |
| *** nicedice has quit IRC | 05:58 | |
| *** gokrokve has quit IRC | 06:02 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Small documentation tweaks https://review.openstack.org/84950 | 06:02 |
| *** jlibosva has joined #openstack-infra | 06:04 | |
| *** denis_makogon has joined #openstack-infra | 06:05 | |
| *** alex-away is now known as asettle | 06:06 | |
| *** yolanda has joined #openstack-infra | 06:06 | |
| tchaypo | dstufft: rebased | 06:07 |
| *** andreaf has quit IRC | 06:08 | |
| *** andreaf2 has quit IRC | 06:12 | |
| *** thuc has joined #openstack-infra | 06:13 | |
| *** ildikov_ has quit IRC | 06:14 | |
| *** wenlock_ has quit IRC | 06:16 | |
| *** thuc has quit IRC | 06:17 | |
| *** _nadya_ has joined #openstack-infra | 06:31 | |
| *** gokrokve has joined #openstack-infra | 06:33 | |
| *** andreaf has joined #openstack-infra | 06:33 | |
| andreaf | jeblair: good morning - I updated https://review.openstack.org/#/c/79314/, thanks for your review. Could you have another look at it? | 06:35 |
| *** matsuhashi has quit IRC | 06:36 | |
| *** mrmartin has joined #openstack-infra | 06:36 | |
| *** CaptTofu has quit IRC | 06:37 | |
| *** CaptTofu has joined #openstack-infra | 06:38 | |
| *** matsuhashi has joined #openstack-infra | 06:39 | |
| *** yolanda has quit IRC | 06:42 | |
| *** CaptTofu has quit IRC | 06:42 | |
| *** gokrokve has quit IRC | 06:43 | |
| *** _nadya_ has quit IRC | 06:44 | |
| *** _nadya_ has joined #openstack-infra | 06:45 | |
| *** sdake has quit IRC | 06:48 | |
| *** _nadya_ has quit IRC | 06:49 | |
| *** tianst20 has joined #openstack-infra | 06:50 | |
| *** alff_ has joined #openstack-infra | 06:51 | |
| *** alff has joined #openstack-infra | 06:51 | |
| *** masayukig has quit IRC | 06:52 | |
| *** jpich has joined #openstack-infra | 06:52 | |
| *** e0ne has joined #openstack-infra | 06:54 | |
| *** VijayTripathi has quit IRC | 06:54 | |
| *** afazekas has joined #openstack-infra | 06:56 | |
| lifeless | clarkb: no leaks so far, will check later still | 06:56 |
| *** mrda is now known as mrda_away | 06:57 | |
| *** asettle is now known as alex-gone | 07:02 | |
| *** zigo has quit IRC | 07:04 | |
| *** sdague has quit IRC | 07:05 | |
| *** sdague has joined #openstack-infra | 07:05 | |
| *** markwash_ has joined #openstack-infra | 07:07 | |
| *** zigo has joined #openstack-infra | 07:08 | |
| *** markwash has quit IRC | 07:08 | |
| *** markwash_ is now known as markwash | 07:08 | |
| *** mikal has quit IRC | 07:10 | |
| *** mikal has joined #openstack-infra | 07:10 | |
| *** mattoliverau has quit IRC | 07:10 | |
| *** ildikov_ has joined #openstack-infra | 07:13 | |
| *** mattoliverau has joined #openstack-infra | 07:14 | |
| *** alff_ has quit IRC | 07:19 | |
| *** alff has quit IRC | 07:19 | |
| *** jgallard has joined #openstack-infra | 07:22 | |
| *** mattoliverau has quit IRC | 07:22 | |
| *** e0ne_ has joined #openstack-infra | 07:25 | |
| *** mattoliverau has joined #openstack-infra | 07:26 | |
| *** che-arne has quit IRC | 07:28 | |
| *** e0ne has quit IRC | 07:29 | |
| *** hashar has joined #openstack-infra | 07:29 | |
| *** marun is now known as marun_afk | 07:29 | |
| *** roeyc has joined #openstack-infra | 07:35 | |
| *** gokrokve has joined #openstack-infra | 07:40 | |
| *** gokrokve has quit IRC | 07:45 | |
| *** mihgen has quit IRC | 07:59 | |
| *** derekh has joined #openstack-infra | 08:05 | |
| *** yassine has joined #openstack-infra | 08:08 | |
| *** locke105 has quit IRC | 08:16 | |
| *** fbo_away is now known as fbo | 08:16 | |
| *** denis_makogon has quit IRC | 08:30 | |
| *** hashar has quit IRC | 08:31 | |
| *** derekh has quit IRC | 08:33 | |
| *** mihgen has joined #openstack-infra | 08:34 | |
| *** dizquierdo has joined #openstack-infra | 08:37 | |
| *** gokrokve has joined #openstack-infra | 08:40 | |
| *** gokrokve has quit IRC | 08:44 | |
| *** pblaho has joined #openstack-infra | 08:45 | |
| *** jp_at_hp has joined #openstack-infra | 08:46 | |
| *** gpocentek has quit IRC | 08:47 | |
| *** gpocentek has joined #openstack-infra | 08:47 | |
| *** roeyc has quit IRC | 08:48 | |
| *** hashar has joined #openstack-infra | 08:52 | |
| *** e0ne_ has quit IRC | 08:54 | |
| *** e0ne has joined #openstack-infra | 08:55 | |
| davidlenwell | that is a new one to me .. I got "unstable" in the gate on a review that was just an rst file.. | 08:55 |
| *** che-arne has joined #openstack-infra | 08:56 | |
| *** Hal has joined #openstack-infra | 09:00 | |
| *** Hal is now known as Guest78329 | 09:00 | |
| Guest78329 | Hi ! Is someone here could help me understand strange dates (uploaded vs updated) on a patchset review : https://review.openstack.org/#/c/28018/. The uploaded date is post updated one. Stange or feature ? | 09:02 |
| *** markmc has joined #openstack-infra | 09:04 | |
| *** yolanda has joined #openstack-infra | 09:05 | |
| *** matsuhashi has quit IRC | 09:07 | |
| *** e0ne_ has joined #openstack-infra | 09:08 | |
| *** jcoufal has quit IRC | 09:09 | |
| *** jcoufal has joined #openstack-infra | 09:10 | |
| *** e0ne has quit IRC | 09:12 | |
| *** matsuhashi has joined #openstack-infra | 09:12 | |
| *** ams0 has joined #openstack-infra | 09:13 | |
| *** morganfainberg is now known as morganfainberg_Z | 09:15 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Fix token validation https://review.openstack.org/84965 | 09:21 |
| *** ociuhandu has quit IRC | 09:23 | |
| *** reed has joined #openstack-infra | 09:25 | |
| SergeyLukjanov | davidlenwell, probably, it was an issue with running job itself, just recheck | 09:28 |
| davidlenwell | I did .. it merged clean .. thanks .. | 09:29 |
| davidlenwell | SergeyLukjanov: sometimes I just make noise when I see something like that .. thanks for responding | 09:29 |
| *** derekh has joined #openstack-infra | 09:33 | |
| SergeyLukjanov | davidlenwell, np, it happens sometimes | 09:34 |
| *** gokrokve has joined #openstack-infra | 09:41 | |
| *** cyeoh has quit IRC | 09:42 | |
| *** cyeoh has joined #openstack-infra | 09:43 | |
| *** gokrokve has quit IRC | 09:45 | |
| *** ociuhandu has joined #openstack-infra | 09:47 | |
| *** yassine has quit IRC | 09:48 | |
| *** matsuhashi has quit IRC | 09:59 | |
| *** matsuhashi has joined #openstack-infra | 09:59 | |
| *** ArxCruz has joined #openstack-infra | 10:01 | |
| *** ArxCruz has quit IRC | 10:01 | |
| Guest78329 | No idea on the origin of theses dates incoherence ? | 10:06 |
| *** fbo is now known as fbo_away | 10:09 | |
| SergeyLukjanov | Guest78329, hmmm, /me looking | 10:13 |
| Guest78329 | SergeyLukjanov: thanks :-) | 10:13 |
| SergeyLukjanov | Guest78329, probably, it's time when quantum -> neutron gerrit projects rename was happened | 10:14 |
| *** wenlock has quit IRC | 10:14 | |
| SergeyLukjanov | Guest78329, yup, here is an example, the same upload date for | 10:15 |
| SergeyLukjanov | https://review.openstack.org/#/c/32922/ | 10:15 |
| Guest78329 | ha, interesting. | 10:16 |
| Guest78329 | Do you think there is a way to retrieve the original upload date ? | 10:16 |
| *** alexpilotti has joined #openstack-infra | 10:17 | |
| SergeyLukjanov | Guest78329, datetime of the first comment should be very near to the upload datetime (same day I think) | 10:18 |
| SergeyLukjanov | Guest78329, I think it's quite normal that it was backported and approved the same day | 10:19 |
| Guest78329 | SergeyLukjanov: you are right, it is the same workaround reviewstats is using for submission time estimation, the first review is usualy Jenkins one. I was looking for a more direct metadata usage. If you think there is no other way to do it via gerrit, I think the first comment date usage is the most effective workaround (another could be using the internal git branch log, exact date, expensive algo) | 10:21 |
| *** chandankumar_ has joined #openstack-infra | 10:22 | |
| Guest78329 | SergeyLukjanov: So do you think there is no other way ? | 10:22 |
| SergeyLukjanov | Guest78329, /me don't see any other option atm | 10:23 |
| Guest78329 | SergeyLukjanov: thank you very much for the advice and the quick response :) | 10:23 |
| SergeyLukjanov | Guest78329, np | 10:24 |
| *** Guest78329 has quit IRC | 10:29 | |
| *** e0ne_ has quit IRC | 10:30 | |
| *** hashar has quit IRC | 10:33 | |
| *** jgallard has quit IRC | 10:37 | |
| sdague | morning SergeyLukjanov | 10:38 |
| SergeyLukjanov | sdague, morning | 10:39 |
| SergeyLukjanov | sdague, how are you? | 10:39 |
| sdague | good, almost time for 2nd cup of coffee | 10:39 |
| sdague | also excited that someone else seems to be using my devstack-vagrant project, because I got a pull request - https://github.com/sdague/devstack-vagrant/pull/1 | 10:40 |
| SergeyLukjanov | oh, I forget about lunch :( | 10:40 |
| SergeyLukjanov | sdague, nice, looking on it | 10:41 |
| SergeyLukjanov | sdague, mmm, multinode devstack locally | 10:41 |
| *** gokrokve has joined #openstack-infra | 10:42 | |
| SergeyLukjanov | sdague, you can setup vagrant proxy caching to speed up re-build - http://fgrehm.viewdocs.io/vagrant-cachier | 10:43 |
| sdague | I usually just build a mostly built box and stick it on my local http server | 10:45 |
| *** gokrokve has quit IRC | 10:46 | |
| sdague | it will build from a pure upstream 12.04 for easy of use, but if recapture the guest after first build then I save about 12 minutes build time | 10:46 |
| SergeyLukjanov | sdague, cool | 10:47 |
| *** Ryan_Lane has quit IRC | 10:48 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/devstack-gate: Prepare for Juno dev https://review.openstack.org/83075 | 10:48 |
| *** matsuhashi has quit IRC | 10:48 | |
| *** reed has quit IRC | 10:48 | |
| SergeyLukjanov | I should rebase it on https://review.openstack.org/#/c/84287/3/devstack-vm-gate.sh | 10:49 |
| SergeyLukjanov | sdague, correct? | 10:49 |
| SergeyLukjanov | and is it ok to approve https://review.openstack.org/#/c/84287/3/devstack-vm-gate.sh? | 10:51 |
| sdague | I believe https://review.openstack.org/#/c/84287/3 is safe | 10:51 |
| sdague | if you double check my logic to make sure that I didn't drop anything, that would be good | 10:52 |
| sdague | SergeyLukjanov: and yes, I think rebasing on that change is the right thing to do | 10:52 |
| SergeyLukjanov | sdague, going to re-check and approve and then rebase | 10:53 |
| sdague | we're still going to have to wait for stable/icehouse devstack before we approve https://review.openstack.org/83075, which is going to be 2 weeks. | 10:53 |
| SergeyLukjanov | sdague, yup | 10:54 |
| SergeyLukjanov | sdague, your patch adds n-cond while there we no n-cond before | 10:55 |
| *** Clabbe has joined #openstack-infra | 10:55 | |
| *** matsuhashi has joined #openstack-infra | 10:55 | |
| sdague | it wasn't in the base list | 10:56 |
| sdague | because it was not in grizzly? | 10:56 |
| sdague | that can't be right | 10:56 |
| SergeyLukjanov | sdague, oh, sorry | 10:56 |
| sdague | yeh, it was at the end of L37 | 10:56 |
| SergeyLukjanov | sdague, it was in the end of line 37 | 10:56 |
| sdague | that was one of the reasons for the variable reorg | 10:57 |
| sdague | so all the services were logically grouped | 10:57 |
| SergeyLukjanov | yeah, that's a good idea | 10:57 |
| SergeyLukjanov | sdague, is it correct that you'd like to remove sysstat and pidstat? | 10:58 |
| sdague | yes | 10:58 |
| sdague | they were deleted from devstack | 10:58 |
| SergeyLukjanov | oh, got it | 10:58 |
| sdague | dstat is just much nicer | 10:59 |
| sdague | and written in python so easy to extend if we want to do better stats | 10:59 |
| SergeyLukjanov | yeah, agreed | 11:00 |
| SergeyLukjanov | it seems really good to use dstat IMO | 11:00 |
| SergeyLukjanov | sdague, looks like it's not good to remove "cinder" from services | 11:02 |
| SergeyLukjanov | [~/workspace/openstack-dev/devstack] ➜ grep -irn "enable.*cinder" . | grep -v .git [master] [15:01:26] | 11:02 |
| SergeyLukjanov | ./exercises/boot_from_volume.sh:45:is_service_enabled cinder || exit 55 | 11:02 |
| SergeyLukjanov | ./exercises/volumes.sh:42:is_service_enabled cinder || exit 55 | 11:02 |
| SergeyLukjanov | ./lib/cinder:163: if is_service_enabled cinder; then | 11:02 |
| SergeyLukjanov | ./lib/nova:471: if is_service_enabled cinder; then | 11:02 |
| SergeyLukjanov | ./stack.sh:727:if is_service_enabled cinder; then | 11:02 |
| SergeyLukjanov | ./stack.sh:1024:if is_service_enabled cinder; then | 11:02 |
| SergeyLukjanov | ./stack.sh:1193:if is_service_enabled cinder; then | 11:02 |
| SergeyLukjanov | ./unstack.sh:133:if is_service_enabled cinder; then | 11:02 |
| sdague | hmmmm | 11:04 |
| sdague | how did it pass? | 11:04 |
| sdague | also, thanks for catching that | 11:04 |
| StevenK | SergeyLukjanov: git grep ? :-0 | 11:08 |
| StevenK | :-) even | 11:08 |
| *** afazekas has quit IRC | 11:09 | |
| *** e0ne has joined #openstack-infra | 11:09 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Correcting the right path to mysql log in Fedora 19 https://review.openstack.org/79967 | 11:09 |
| *** e0ne_ has joined #openstack-infra | 11:11 | |
| SergeyLukjanov | StevenK, I'm always forgetting about git grep when running grep ;) | 11:12 |
| *** e0ne__ has joined #openstack-infra | 11:13 | |
| *** e0ne_ has quit IRC | 11:13 | |
| *** e0ne has quit IRC | 11:13 | |
| *** matsuhashi has quit IRC | 11:15 | |
| lifeless | Clabbe: still no leaks, so I think thats good | 11:15 |
| SergeyLukjanov | sdague, there are the same usages for nova and glance | 11:15 |
| *** matsuhashi has joined #openstack-infra | 11:15 | |
| sdague | SergeyLukjanov: is_service_enabled nova should do the right thing | 11:17 |
| sdague | I guess we just don't have the is_cinder_enabled script yet | 11:17 |
| *** hashar has joined #openstack-infra | 11:17 | |
| sdague | I'll fix d-g | 11:17 |
| *** e0ne__ has quit IRC | 11:18 | |
| *** matsuhashi has quit IRC | 11:19 | |
| *** yassine has joined #openstack-infra | 11:20 | |
| *** matsuhashi has joined #openstack-infra | 11:21 | |
| *** yjiang has quit IRC | 11:21 | |
| *** zhiyan_ is now known as zhiyan | 11:22 | |
| *** afazekas has joined #openstack-infra | 11:24 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: normalize the services for a post grizzly world https://review.openstack.org/84287 | 11:25 |
| sdague | SergeyLukjanov: there we go | 11:26 |
| sdague | thanks for the catch | 11:26 |
| SergeyLukjanov | sdague, np, I'm glad to be useful here ;) | 11:26 |
| anteaya | mattoliverau: I like the way you think, and I can get behind a tent on a barge | 11:27 |
| *** jcoufal has quit IRC | 11:31 | |
| sdague | anteaya: I clearly missed an interesting conversation :) | 11:32 |
| *** jgallard has joined #openstack-infra | 11:35 | |
| *** jgallard has quit IRC | 11:36 | |
| *** jgallard has joined #openstack-infra | 11:36 | |
| *** mbacchi has joined #openstack-infra | 11:38 | |
| *** dcramer_ has quit IRC | 11:41 | |
| *** e0ne has joined #openstack-infra | 11:42 | |
| *** gokrokve has joined #openstack-infra | 11:43 | |
| *** mbacchi has quit IRC | 11:44 | |
| *** rfolco has joined #openstack-infra | 11:45 | |
| *** HenryG has joined #openstack-infra | 11:47 | |
| *** gokrokve has quit IRC | 11:48 | |
| *** flaper87|afk is now known as flaper87 | 11:48 | |
| *** talluri has quit IRC | 11:48 | |
| anteaya | sdague: 'tis in the backscroll | 11:51 |
| anteaya | one way or another, pencil some part of austrailia into your upcoming iternerary | 11:51 |
| anteaya | clarkb wants Sydney for M | 11:52 |
| anteaya | which might end up being a tent on a barge | 11:52 |
| anteaya | which has a nice sound to it | 11:52 |
| *** matsuhashi has quit IRC | 11:52 | |
| anteaya | and mattoliverau figures if we hit oz for mid-cycle we might as well rent a waterpark for the day | 11:53 |
| sdague | nice | 11:53 |
| sdague | Nov 2016 in Sydney would be cool | 11:54 |
| sdague | Nov 2015 in Sydney would be cool | 11:54 |
| sdague | stupid maths | 11:54 |
| anteaya | we could do it twice in a row | 11:55 |
| anteaya | double header | 11:55 |
| *** mbacchi has joined #openstack-infra | 11:55 | |
| *** yaguang has quit IRC | 11:56 | |
| *** weshay has joined #openstack-infra | 11:56 | |
| *** mriedem has joined #openstack-infra | 12:00 | |
| *** dmsimard has joined #openstack-infra | 12:05 | |
| dmsimard | Hi guys | 12:06 |
| dmsimard | http://logs.openstack.org/79/81179/6/check/gate-puppet-ceph-puppet-lint/a7d1339/console.html | 12:09 |
| dmsimard | "http://logs.openstack.org/79/81179/6/check/gate-puppet-ceph-puppet-lint/a7d1339/console.html#_2014-04-02_03_02_52_942 | no such file to load -- rspec-system/rake_task" | 12:09 |
| *** gokrokve has joined #openstack-infra | 12:09 | |
| dmsimard | Yet the test passed ? | 12:09 |
| dmsimard | It should have failed, no ? | 12:09 |
| *** yamahata has quit IRC | 12:10 | |
| *** StevenK_ has joined #openstack-infra | 12:10 | |
| *** fbo_away is now known as fbo | 12:10 | |
| *** mrodden has quit IRC | 12:10 | |
| *** mrodden has joined #openstack-infra | 12:10 | |
| *** StevenK has quit IRC | 12:10 | |
| *** gokrokve has quit IRC | 12:13 | |
| *** persia has quit IRC | 12:15 | |
| *** persia has joined #openstack-infra | 12:15 | |
| *** persia is now known as Guest57160 | 12:15 | |
| *** ArxCruz has joined #openstack-infra | 12:20 | |
| *** pdmars has joined #openstack-infra | 12:21 | |
| anteaya | hi dmsimard I'm not familar with the expectations of that test, you might try the core for that project: https://review.openstack.org/#/admin/groups/226,members | 12:21 |
| *** pdmars has quit IRC | 12:21 | |
| dmsimard | I'm a core :) | 12:22 |
| *** pdmars has joined #openstack-infra | 12:22 | |
| dmsimard | I'll check the test definitions more thoroughly | 12:22 |
| *** david-lyle has quit IRC | 12:23 | |
| anteaya | okay | 12:25 |
| *** aysyd has joined #openstack-infra | 12:26 | |
| *** jhesketh has quit IRC | 12:26 | |
| *** yolanda has quit IRC | 12:26 | |
| *** che-arne has quit IRC | 12:30 | |
| *** che-arne has joined #openstack-infra | 12:31 | |
| *** che-arne has quit IRC | 12:32 | |
| *** che-arne has joined #openstack-infra | 12:33 | |
| fungi | dmsimard: you probably want to prepend #!/bin/bash -xe on the builder at http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/macros.yaml#n77 | 12:35 |
| fungi | assuming the other commands in the builder are expected to exit 0 under normal conditions as well | 12:35 |
| *** Guest57160 has quit IRC | 12:36 | |
| *** Guest57160 has joined #openstack-infra | 12:36 | |
| *** Guest57160 is now known as persia | 12:36 | |
| fungi | dmsimard: though since rake lint is the last line of that shell snippet, it shouldn't be necessary to catch that particular failure | 12:37 |
| fungi | dmsimard: what are the chances that when rake lint can't find rspec-system/rake_task it exits 0? | 12:37 |
| fungi | that would certainly explain the behavior you're seeing | 12:37 |
| *** che-arne has quit IRC | 12:38 | |
| dmsimard | yeah, I'll talk with mgagne or bodepd who are more familiar with that particular job, see what we can do. It's a common job so I have to be especially careful :) | 12:38 |
| *** miqui has joined #openstack-infra | 12:40 | |
| *** yolanda has joined #openstack-infra | 12:40 | |
| *** xchu_ has joined #openstack-infra | 12:42 | |
| *** ameade has joined #openstack-infra | 12:46 | |
| *** nosnos has quit IRC | 12:50 | |
| *** eharney has joined #openstack-infra | 12:53 | |
| openstackgerrit | A change was merged to openstack-infra/config: Drop max-servers to 40 for tripleo HP region. https://review.openstack.org/84903 | 12:53 |
| *** thuc has joined #openstack-infra | 12:54 | |
| *** thuc_ has joined #openstack-infra | 12:54 | |
| *** dprince has joined #openstack-infra | 12:55 | |
| *** che-arne has joined #openstack-infra | 12:58 | |
| *** thuc has quit IRC | 12:58 | |
| *** yaguang has joined #openstack-infra | 13:00 | |
| *** mancdaz has quit IRC | 13:01 | |
| *** mancdaz has joined #openstack-infra | 13:01 | |
| *** lcostantino has joined #openstack-infra | 13:02 | |
| *** StevenK_ is now known as StevenK | 13:02 | |
| *** adalbas has quit IRC | 13:03 | |
| *** adalbas has joined #openstack-infra | 13:04 | |
| sdague | just when I thought I understood the oslo.config breaks, I found a new edge condition | 13:05 |
| *** dims_ has quit IRC | 13:07 | |
| *** maxbit has joined #openstack-infra | 13:07 | |
| fungi | sdague: oslo.config has sharp edges in more than 3 dimensions | 13:07 |
| fungi | we need a hyperspatial map to explore them adequately | 13:08 |
| *** dcramer_ has joined #openstack-infra | 13:09 | |
| *** zehicle_at_dell has quit IRC | 13:09 | |
| *** gokrokve has joined #openstack-infra | 13:10 | |
| sdague | I totally need an occulus rift debugger for them | 13:10 |
| *** jeckersb_gone is now known as jeckersb | 13:10 | |
| *** changbl has quit IRC | 13:11 | |
| *** gokrokve has quit IRC | 13:14 | |
| *** julim has joined #openstack-infra | 13:15 | |
| fungi | i was thinking a skulljack--after all your eyes are really only interpreting 3 dimensions courtesy of parallax between a pair of two-dimensional surfaces | 13:17 |
| *** che-arne has quit IRC | 13:19 | |
| *** chandankumar_ has quit IRC | 13:19 | |
| *** skraynev is now known as skraynev_afk | 13:19 | |
| *** Guest78329 has joined #openstack-infra | 13:19 | |
| *** dims_ has joined #openstack-infra | 13:20 | |
| *** rfolco has quit IRC | 13:20 | |
| *** zns has joined #openstack-infra | 13:21 | |
| *** rfolco has joined #openstack-infra | 13:22 | |
| *** signed8bit has joined #openstack-infra | 13:23 | |
| *** weshay has quit IRC | 13:27 | |
| *** weshay has joined #openstack-infra | 13:30 | |
| *** freyes has joined #openstack-infra | 13:30 | |
| *** thomasem has joined #openstack-infra | 13:31 | |
| openstackgerrit | Ilya Shakhat proposed a change to openstack-infra/config: Add new stackforge project DriverLog https://review.openstack.org/85033 | 13:37 |
| *** gondoi has quit IRC | 13:40 | |
| *** thuc_ has quit IRC | 13:40 | |
| *** gondoi has joined #openstack-infra | 13:40 | |
| *** thuc has joined #openstack-infra | 13:41 | |
| dhellmann | sdague: what's this one? | 13:42 |
| *** nkinder has quit IRC | 13:45 | |
| *** thuc has quit IRC | 13:45 | |
| *** jgrimm has quit IRC | 13:45 | |
| *** mihgen has quit IRC | 13:46 | |
| sdague | the setup_install change from yesterday | 13:47 |
| sdague | seems to cause a grenade issue | 13:48 |
| sdague | because it's not doing pip install -U | 13:48 |
| sdague | only pip install | 13:48 |
| sdague | https://review.openstack.org/#/c/85027/ | 13:48 |
| *** hughsaunders has quit IRC | 13:50 | |
| *** ameade has quit IRC | 13:52 | |
| *** sdake__ is now known as sdake | 13:56 | |
| *** hughsaunders has joined #openstack-infra | 13:57 | |
| *** mwagner_lap has joined #openstack-infra | 14:01 | |
| *** german_ has joined #openstack-infra | 14:01 | |
| ruhe | folks, this change https://review.openstack.org/#/c/80690/ (If dependency not in global reqs exit with a 1). is there any way to disable that check for particular gate job? | 14:01 |
| flaper87 | how does one enable SSL in devstack ? | 14:02 |
| *** ominakov has joined #openstack-infra | 14:05 | |
| openstackgerrit | Ilya Shakhat proposed a change to openstack-infra/config: Add new stackforge project DriverLog https://review.openstack.org/85033 | 14:05 |
| *** thuc has joined #openstack-infra | 14:06 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/elastic-recheck: Add query for a heat db error https://review.openstack.org/85039 | 14:07 |
| *** che-arne has joined #openstack-infra | 14:09 | |
| mtreinish | SergeyLukjanov: on https://review.openstack.org/#/c/84840/ the duplication is intentional | 14:10 |
| mtreinish | look at the follow-up patch which fixes the names for the duplicate jobs | 14:10 |
| *** gokrokve has joined #openstack-infra | 14:11 | |
| *** wenlock has joined #openstack-infra | 14:14 | |
| *** xchu_ has quit IRC | 14:15 | |
| andreaf | sdague, jeblair: could you have a look at https://review.openstack.org/#/c/79314/ - I'd like to start running the keystone v3 experimental checks | 14:15 |
| *** gokrokve has quit IRC | 14:15 | |
| *** thedodd has joined #openstack-infra | 14:18 | |
| *** wenlock has quit IRC | 14:18 | |
| dhellmann | the new job definitions for oslo mean we're running gate-{name}-docs, but some of the libs have their docs hosted on rtfd.org | 14:19 |
| dhellmann | I could fix it by copying the job definitions to all of those libs, or making a new template in layout.yaml and moving the docs jobs there | 14:19 |
| dhellmann | any preference for which approach is right? | 14:19 |
| dhellmann | maybe a dev-docs template? | 14:20 |
| *** pblaho has quit IRC | 14:21 | |
| jeblair | dhellmann: all openstack projects have docs hosted on docs.o.o/developer, so we should have gate-name-docs (which just verifies that they can build), but we should also have name-docs which publishes them there | 14:21 |
| openstackgerrit | Ilya Shakhat proposed a change to openstack-infra/config: Add new stackforge project DriverLog https://review.openstack.org/85033 | 14:21 |
| dhellmann | jeblair: ok, so are we still in requirements freeze? I would need to add Pillow and maybe some other libs. The stevedore docs use graphviz, too. | 14:22 |
| dhellmann | I suppose I could comment out the diagrams for now | 14:22 |
| jeblair | andreaf: can you change "check-" to "{pipeline}-" in the job names in the jjb definition? | 14:22 |
| jeblair | dhellmann: tbh, i don't know if we're in "test-requirements" freeze or if that's even a thing... :) ttx? | 14:23 |
| andreaf | jeblair: but that would run in the experimental pipeline, is that ok? I checked other experimental jobs and they use check instead of {pipeline}# | 14:23 |
| *** thuc_ has joined #openstack-infra | 14:24 | |
| *** markmcclain has joined #openstack-infra | 14:24 | |
| jeblair | andreaf: it just defines the jobs, it doesn't necessarily run them | 14:24 |
| *** jgrimm has joined #openstack-infra | 14:24 | |
| andreaf | jeblair: do I have to define experimental-.... jobs then? | 14:25 |
| *** vhoward- has joined #openstack-infra | 14:25 | |
| jeblair | andreaf: nope, just change "check-" to "{pipeline}-" and the template invocation will create "check-" and "gate-" versions of the job. you can ignore the gate version and leave "check-" as the job that's invoked in layout.yaml | 14:25 |
| andreaf | jeblair: right, got it, thanks | 14:26 |
| *** chandan_kumar has quit IRC | 14:27 | |
| *** chandan_kumar has joined #openstack-infra | 14:27 | |
| *** gokrokve has joined #openstack-infra | 14:27 | |
| *** thuc has quit IRC | 14:28 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack/requirements: Add Pillow for stevedore doc build https://review.openstack.org/85047 | 14:28 |
| *** thuc_ has quit IRC | 14:28 | |
| ttx | jeblair: grey area. makes sense to freeze them as well since good packagers should run the tests so need to have test-deps packaged | 14:29 |
| *** chandan_kumar has quit IRC | 14:30 | |
| jeblair | ttx: thanks | 14:30 |
| dhellmann | ttx, jeblair : can I at least make that job non-voting for now? it's brand new -- yesterday | 14:30 |
| *** afazekas has quit IRC | 14:30 | |
| dhellmann | otherwise we won't be able to do any work on the cross-project gating jobs | 14:30 |
| ttx | dhellmann: sure, don't want that to block us too much | 14:31 |
| dhellmann | ok, good, I'll do that for now | 14:31 |
| dhellmann | the other changes are up for review, too, but those can wait | 14:32 |
| *** alff has joined #openstack-infra | 14:33 | |
| *** alff_ has joined #openstack-infra | 14:33 | |
| *** esker has joined #openstack-infra | 14:33 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack-infra/config: Add experimental keystonev3 jobs https://review.openstack.org/79314 | 14:35 |
| andreaf | jeblair: done. thanks - I think I finally understood jjb :P | 14:36 |
| *** CaptTofu has joined #openstack-infra | 14:36 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Add the second tripleo provider https://review.openstack.org/83057 | 14:37 |
| *** nkinder has joined #openstack-infra | 14:38 | |
| *** weshay has quit IRC | 14:38 | |
| fungi | ruhe: doesn't look like there's a toggle to skip that behavior--where is it exhibiting an issue? | 14:39 |
| *** markwash has quit IRC | 14:39 | |
| fungi | andreaf: see my comments on 79314 | 14:39 |
| fungi | sorry i didn't catch you fast enough before you uploaded another patchset | 14:39 |
| *** wenlock has joined #openstack-infra | 14:40 | |
| andreaf | fungi: thanks for the review :) I'll fix it right away | 14:41 |
| fungi | andreaf: you're welcome! | 14:42 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Disable gate-stevedore-docs https://review.openstack.org/85055 | 14:43 |
| dhellmann | jeblair, ttx: ^^ | 14:44 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack-infra/config: Add experimental keystonev3 jobs https://review.openstack.org/79314 | 14:44 |
| andreaf | fungi, jeblair: ^^^ new patch-set up | 14:45 |
| *** rlandy has joined #openstack-infra | 14:45 | |
| openstackgerrit | Brant Knudson proposed a change to openstack/requirements: Remove pam https://review.openstack.org/85058 | 14:49 |
| *** ameade has joined #openstack-infra | 14:50 | |
| *** vhoward- has left #openstack-infra | 14:51 | |
| *** yaguang has quit IRC | 14:52 | |
| *** mfer has joined #openstack-infra | 14:52 | |
| dhellmann | ok, now the requirements check for cliff is complaining about duplicate requirements "in master" https://jenkins02.openstack.org/job/gate-cliff-requirements/1/console | 14:52 |
| dhellmann | I've submitted a change to remove the dupes, but it won't pass because the original files are broken? | 14:53 |
| *** pcrews_ has joined #openstack-infra | 14:55 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Disable broken oslo jobs https://review.openstack.org/85055 | 14:56 |
| dhellmann | updated the previous change to disable the cliff requirements job, too ^^ | 14:56 |
| dhellmann | starting new libs is going to be a lot simpler than adopting existing projects like cliff and stevedore :-) | 14:57 |
| *** thuc has joined #openstack-infra | 14:58 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Small documentation tweaks https://review.openstack.org/84950 | 14:58 |
| *** thuc_ has joined #openstack-infra | 14:58 | |
| *** mihgen has joined #openstack-infra | 14:59 | |
| *** rlandy has quit IRC | 15:00 | |
| *** rlandy has joined #openstack-infra | 15:00 | |
| *** mgagne1 has quit IRC | 15:00 | |
| fungi | dhellmann: we probably need to do something similar to https://review.openstack.org/84590 | 15:01 |
| dhellmann | fungi: I *thought* I had seen something about that recently | 15:02 |
| *** thuc has quit IRC | 15:02 | |
| dhellmann | fungi: which is likely to land sooner? :-) | 15:02 |
| fungi | dhellmann: looks like there's already an ignore_dups flag, so maybe just add it to the correct read() call | 15:02 |
| dhellmann | fungi: ok, I'll see about adding that | 15:03 |
| fungi | dhellmann: i think it's just branch_reqs.read_all_requirements(ignore_dups=True) | 15:04 |
| fungi | oh, except that's not passed through from read_all_requirements() | 15:05 |
| fungi | dhellmann: i'll just amend my change to do that bit from the strict flag | 15:05 |
| *** hashar has quit IRC | 15:05 | |
| dhellmann | fungi: ok, cool | 15:05 |
| *** weshay has joined #openstack-infra | 15:08 | |
| *** julienvey has left #openstack-infra | 15:08 | |
| *** atiwari has joined #openstack-infra | 15:11 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Requirements check should tolerate existing cruft https://review.openstack.org/84590 | 15:11 |
| fungi | dhellmann: ^ (added a little more explanation to the commit message too) | 15:11 |
| *** dguerri_ is now known as _dguerri | 15:11 | |
| *** _dguerri is now known as dguerri_ | 15:12 | |
| dhellmann | fungi: why do we run those tests against master before merging in the proposed change? | 15:12 |
| fungi | dhellmann: due to an oversight... looks like slightly overzealous drying | 15:13 |
| dhellmann | you added strict=True on line 98, which I think is still going to cause cliff's job to fail | 15:14 |
| fungi | the same function gets used for reading the old and new copies to generate the dict | 15:14 |
| *** david-lyle has joined #openstack-infra | 15:14 | |
| fungi | but the style checks are performed in the read function | 15:14 |
| dhellmann | maybe I'm confused about the initial state of the repo | 15:14 |
| fungi | so this change turns the style checks off while reading the original | 15:14 |
| dhellmann | so the original values are read second? on line 103? | 15:15 |
| *** VijayTripathi has joined #openstack-infra | 15:15 | |
| fungi | s/generate the dict/generate the DIFF/ | 15:15 |
| fungi | dhellmann: yes | 15:15 |
| dhellmann | ah, ok, I had it backwards :-) | 15:15 |
| fungi | head_reqs is from the git fetch head (your proposed change) and branch_reqs is from the target branch tip | 15:15 |
| fungi | i agree the variable names aren't exceedingly elucidating | 15:16 |
| dhellmann | ok, it makes sense now, I was just thinking of it happening in the other order and mis-reading the code as a result | 15:16 |
| dhellmann | +1 | 15:16 |
| fungi | perhaps since i've sussed out the spaghetti there, i should add some comments | 15:16 |
| fungi | doing that now | 15:16 |
| * dhellmann nods | 15:16 | |
| fungi | because that makes at least three people i had to explain the code too, so i think that means it's not readable as is | 15:17 |
| fungi | er, to | 15:17 |
| fungi | took me a while of staring at it as well | 15:17 |
| dhellmann | my future self is constantly battling with my past self | 15:18 |
| dhellmann | and of course that leaves my present self stuck in the middle | 15:19 |
| *** zns has quit IRC | 15:21 | |
| *** sdake_ has joined #openstack-infra | 15:22 | |
| *** sdake_ has quit IRC | 15:22 | |
| *** sdake_ has joined #openstack-infra | 15:22 | |
| *** zns has joined #openstack-infra | 15:24 | |
| *** zhiyan is now known as zhiyan_ | 15:25 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Requirements check should tolerate existing cruft https://review.openstack.org/84590 | 15:26 |
| fungi | dhellmann: ^ now with comments![tm] | 15:26 |
| dhellmann | fungi: nice! | 15:28 |
| *** yamahata has joined #openstack-infra | 15:28 | |
| fungi | hopefully nobody need explain that script ever again | 15:29 |
| dhellmann | heh | 15:30 |
| *** ams0 has quit IRC | 15:30 | |
| dhellmann | fungi: if you think we can land your change today, I can keep the cliff requirements job voting | 15:30 |
| sandywalsh | hey guys, when adding people to a gerrit group, is there anything required for them other than a launchpad account? | 15:31 |
| fungi | dhellmann: if jeblair's cool with it, or mordred/clarkb/SergeyLukjanov happen to be around, it seems likely | 15:31 |
| sandywalsh | I have a user I'm trying to add that isn't getting found. | 15:31 |
| fungi | sandywalsh: their lp account name may not be the same as the gerrit account name they're authenticating with it | 15:32 |
| *** mgagne has joined #openstack-infra | 15:32 | |
| dhellmann | fungi: ok, I'll wait, and if yours merges I'll update my change so it only turns off the stevedore job that wasn't working | 15:33 |
| fungi | sandywalsh: gerrit only relies on lp accounts for openid authentication, and can import some details initially from the openid metadata the first time they log in | 15:33 |
| sandywalsh | fungi: k, I'll verify that. Thanks | 15:33 |
| fungi | sandywalsh: but some details (name, e-mail addresses, et cetera) can change in gerrit later | 15:33 |
| sandywalsh | fungi: gotcha | 15:33 |
| fungi | sandywalsh: similarly, they may make changes to lp after that which aren't reflected in gerrit | 15:33 |
| fungi | sandywalsh: and if they've never logged into gerrit, they won't necessarily have an account at all (that goes for pretty much any new accounts in the past year or so) | 15:34 |
| *** mgagne1 has joined #openstack-infra | 15:34 | |
| *** mgagne1 has quit IRC | 15:34 | |
| *** mgagne1 has joined #openstack-infra | 15:34 | |
| sandywalsh | fungi: that's what I suspect the problem is. I don't think they've logged in. | 15:35 |
| *** MarkAtwood has quit IRC | 15:35 | |
| fungi | if they never log into gerrit, probably no need for them to belong to any groups in it ;) | 15:35 |
| *** rcleere has joined #openstack-infra | 15:35 | |
| sandywalsh | yeah, that's for sure | 15:36 |
| *** mgagne has quit IRC | 15:38 | |
| *** CaptTofu has quit IRC | 15:40 | |
| *** andreaf has quit IRC | 15:41 | |
| derekh | Hi, just wondering why the check-tripleo-overcloud-f20 is coming is as NOT_REGISTERED? do I need to do anything else to start the job running? e.g. https://review.openstack.org/#/c/84876/ | 15:42 |
| *** che-arne has quit IRC | 15:43 | |
| fungi | derekh: i think it means we're not getting (nor never have while the current jenkins master servers were running) any nodes building successfully for the node label that job wants | 15:44 |
| fungi | derekh: looks like we have no nodepool images for it | 15:45 |
| *** zehicle has joined #openstack-infra | 15:45 | |
| fungi | checking image logs now to see why. probably never built any successfully since the last time that cloud was rebuilt | 15:45 |
| derekh | fungi: ok, will take a look, the template images seem to be there at least, | 15:45 |
| derekh | fungi: tripleo-f20-1396318442.template.openstack.org | 15:46 |
| fungi | derekh: the base images? | 15:46 |
| fungi | oh, the template instance | 15:46 |
| *** jgriffith has quit IRC | 15:47 | |
| fungi | that template was created 02:14:02 utc tuesday | 15:47 |
| fungi | probably in the midst of the provider issues | 15:47 |
| fungi | i'll blow it away--it's a sign that the image build which was initiated on that try never completed | 15:48 |
| fungi | nodepool may have been restarted in the middle of it or something | 15:48 |
| derekh | fungi: ahh ok, thanks | 15:48 |
| fungi | oh... i was mis-grepping the lists. f20 not fedora | 15:50 |
| fungi | so we do have images, (from a couple days ago) just no new nodes building from them | 15:50 |
| *** zns has joined #openstack-infra | 15:51 | |
| *** flaper87 is now known as flaper87|afk | 15:52 | |
| SergeyLukjanov | morning folks | 15:52 |
| *** jlibosva has quit IRC | 15:52 | |
| *** dguerri_ is now known as _dguerri | 15:53 | |
| *** zns has quit IRC | 15:53 | |
| SergeyLukjanov | fungi, dhellmann, re https://review.openstack.org/#/c/84590/4: IIRC dup requirements will break pip install -r ... | 15:53 |
| fungi | SergeyLukjanov: agreed, but does that job try to pip install -r? | 15:53 |
| *** zns has joined #openstack-infra | 15:53 | |
| *** zehicle has quit IRC | 15:54 | |
| fungi | SergeyLukjanov: anyway, the point is to allow people to submit changes which remove the dupe requirements | 15:54 |
| dhellmann | SergeyLukjanov: right, we have dupes now but can't remove them because the job prevents the patch from landing | 15:54 |
| fungi | SergeyLukjanov: if the job bails on existing duplicated already on the branch without testing the proposed fix, then it leaves the problem unfixable | 15:54 |
| dhellmann | SergeyLukjanov: also, I think pip doesn't complain unless the requirements are different | 15:54 |
| SergeyLukjanov | fungi, agreed | 15:55 |
| *** jgallard has quit IRC | 15:55 | |
| fungi | SergeyLukjanov: so the intended outcome of this change is to not let new changes land *unless* they fix the dupes, trailing newlines, et cetera, but at least allow them to be fixed | 15:55 |
| *** che-arne has joined #openstack-infra | 15:55 | |
| SergeyLukjanov | fungi, sounds reasonable | 15:55 |
| fungi | the way the script is currently written, you can't merge a fix for those problems | 15:56 |
| *** _dguerri is now known as dguerri_ | 15:56 | |
| SergeyLukjanov | fungi, +2 from me | 15:57 |
| SergeyLukjanov | dhellmann, ^^ | 15:57 |
| dhellmann | SergeyLukjanov: thanks! | 15:57 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add subway map to status page https://review.openstack.org/84314 | 15:58 |
| *** jgriffith has joined #openstack-infra | 15:58 | |
| *** zns_ has joined #openstack-infra | 15:58 | |
| fungi | derekh: so looking in the debug logs, nodepool sees a deficit/allocation request for tripleo-f20 nodes, but isn't trying to build any. current suspicions are misconfiguration on that label in nodepool.yaml, or maybe something making the snapshot images unusable for them which isn't being noted in the log | 15:59 |
| *** changbl has joined #openstack-infra | 15:59 | |
| fungi | derekh: also it's failing to build new images for those because of a failure to add a floating-ip to the template instance, not sure whether it's related | 15:59 |
| *** mrmartin has quit IRC | 15:59 | |
| derekh | fungi: ok, just on a call at the moment for will look through some logs once I'm off | 16:00 |
| *** alff_ has quit IRC | 16:00 | |
| *** alff has quit IRC | 16:00 | |
| *** jlibosva has joined #openstack-infra | 16:01 | |
| *** marun_afk is now known as marun | 16:01 | |
| *** zns has quit IRC | 16:01 | |
| fungi | derekh: i don't think we're hitting our floating-ip quota because nova floating-ip-list only shows 33 entries | 16:01 |
| *** thuc_ has quit IRC | 16:02 | |
| *** thuc has joined #openstack-infra | 16:02 | |
| *** andreaf has joined #openstack-infra | 16:03 | |
| *** mdenny has quit IRC | 16:05 | |
| *** mdenny has joined #openstack-infra | 16:05 | |
| derekh | fungi: does the IP fail to get added or can you not ssh to the IP? if its the later I think this is an issue lifeless was looking at where the nic's are being added in the wrong order | 16:05 |
| *** MarkAtwood has joined #openstack-infra | 16:06 | |
| *** thuc has quit IRC | 16:06 | |
| fungi | derekh: not sure. i went ahead and blew away the existing f20 images in case there was something about them causing nodepoold not to want to use them for new nodes. nodepoold is now trying to build another image--we'll see if it continues to hit any issues | 16:07 |
| SergeyLukjanov | fungi, could you please take a look at https://review.openstack.org/#/c/84503/? | 16:07 |
| jeblair | fungi: is there a label for them? | 16:07 |
| fungi | jeblair: there is | 16:07 |
| SergeyLukjanov | fungi, +2 from jenkins, but it's not merged | 16:08 |
| fungi | jeblair: i can see in the config the label which the jobs want is represented, refers to the image name (which is the same as the label name in this case) and the debug logs report a deficit/allocation request for the nodes but wasn't trying to build any from the images it had | 16:09 |
| fungi | SergeyLukjanov: that's definitely weird. i'll check zuul's log for that merge | 16:10 |
| SergeyLukjanov | fungi, thank you! | 16:10 |
| *** melwitt has joined #openstack-infra | 16:11 | |
| jeblair | fungi: ok; if it doesn't clear up soon there may be an allocator bug; let me know if you want me to look into it | 16:11 |
| *** freyes has quit IRC | 16:12 | |
| fungi | SergeyLukjanov: it's wip | 16:12 |
| fungi | i missed it when i first looked over the change | 16:12 |
| SergeyLukjanov | fungi, oh, shame on me... | 16:12 |
| fungi | SergeyLukjanov: wip changes can't merge | 16:12 |
| SergeyLukjanov | fungi, do I need to re-approve it? | 16:13 |
| *** krtaylor has quit IRC | 16:13 | |
| fungi | jeblair: agreed, i just wanted to rule out any fallout from the provider outages/bugs before looking harder at it being an allocator logic problem | 16:13 |
| fungi | SergeyLukjanov: i would un-wip, then remove and re-add approval | 16:14 |
| SergeyLukjanov | fungi, I already unwiped it, it's my change :) | 16:14 |
| fungi | SergeyLukjanov: and if it still doesn't work, may need to delete the jenkins crvw=+2 change and reapprove again (as a gerrit admin you should be able to delete other votes) | 16:14 |
| fungi | er, the vfry=+2 | 16:15 |
| SergeyLukjanov | fungi, I'm afraid that I'm not gerrit admin ;( | 16:15 |
| fungi | oh, right, i can delete it if the reapproval isn't working | 16:15 |
| SergeyLukjanov | fungi, anyway, everything is ok, it's in zuul pipeline now | 16:15 |
| fungi | great! | 16:15 |
| SergeyLukjanov | fungi, re-approve is working ok | 16:15 |
| fungi | then no bug there to fix | 16:16 |
| openstackgerrit | Ilya Shakhat proposed a change to openstack-infra/config: Add new stackforge project DriverLog (driver verification) https://review.openstack.org/85033 | 16:16 |
| fungi | derekh: at least we're past the floating-ip issue for the image build... it's currently puppeting the new template instance | 16:16 |
| *** talluri has joined #openstack-infra | 16:17 | |
| SergeyLukjanov | fungi, looks like it works because of https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml#L68 | 16:17 |
| fungi | yep, agreed. i think we had a bug there at one point and then fixed it | 16:17 |
| *** maxbit has quit IRC | 16:18 | |
| *** maxbit has joined #openstack-infra | 16:19 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add subway map to status page https://review.openstack.org/84314 | 16:21 |
| *** fbo is now known as fbo_away | 16:26 | |
| *** yassine has quit IRC | 16:26 | |
| *** locke105 has joined #openstack-infra | 16:27 | |
| *** jpich has quit IRC | 16:27 | |
| clarkb | morning | 16:27 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix jshint linting errors https://review.openstack.org/84905 | 16:28 |
| *** zns_ has quit IRC | 16:28 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix up filter issues introduced by jshint changes https://review.openstack.org/84112 | 16:28 |
| jeblair | i'm at a dead end with https://review.openstack.org/#/c/84314/ i don't know how to undo something bootstrap has done. i'd personally rather just not use bootstrap so it's pretty hard for me to work up the enthusiasm to go any further. | 16:28 |
| *** mrodden has quit IRC | 16:29 | |
| *** maxbit has quit IRC | 16:29 | |
| *** thuc_ has joined #openstack-infra | 16:30 | |
| clarkb | fungi: dhellmann have a link for the cliff thing? fwiw I intend on doing my best to dig into reviews today | 16:30 |
| jeblair | also, no one from wikimedia ever reviewed those changes :( | 16:30 |
| pleia2 | good morning | 16:32 |
| zaro | morning | 16:33 |
| dhellmann | clarkb: https://review.openstack.org/#/c/84590/ | 16:33 |
| dhellmann | clarkb: also https://review.openstack.org/#/c/85055/ | 16:34 |
| jeblair | sdague: when you have a sec; can you see if you need to add anything to this thread: Subject: Re: [OpenStack-Infra] Multi-node aspects of the infra | 16:36 |
| *** bhuvan has joined #openstack-infra | 16:36 | |
| *** ominakov has quit IRC | 16:37 | |
| *** thuc has joined #openstack-infra | 16:38 | |
| *** thuc_ has quit IRC | 16:41 | |
| clarkb | dhellmann: I just approved 84590, should 85055 be ammended to only make stevedore non voting? | 16:46 |
| clarkb | dhellmann: or would you prefer to make them both non voting at this time regardless of 84590 merging? | 16:47 |
| dhellmann | clarkb: let's go ahead and make them both non-voting for now -- I'll fix cliff up and then make it voting again in the next day or two | 16:47 |
| clarkb | ok | 16:48 |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added recursive option https://review.openstack.org/85106 | 16:48 |
| openstackgerrit | A change was merged to openstack-infra/config: Requirements check should tolerate existing cruft https://review.openstack.org/84590 | 16:49 |
| *** e0ne has quit IRC | 16:49 | |
| *** e0ne has joined #openstack-infra | 16:50 | |
| *** krtaylor has joined #openstack-infra | 16:50 | |
| openstackgerrit | A change was merged to openstack-infra/config: Disable broken oslo jobs https://review.openstack.org/85055 | 16:52 |
| dhellmann | thanks, clarkb | 16:52 |
| *** alexpilotti has quit IRC | 16:53 | |
| *** e0ne has quit IRC | 16:54 | |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added recursive option https://review.openstack.org/85106 | 16:55 |
| *** Ryan_Lane has joined #openstack-infra | 16:57 | |
| ruhe | fungi: yes it does. we've recently switched murano tests to the openstack ci gate. but we still have one depedency not present in global-requirements. and it'll take sufficient amount of time to add it to global-requirements | 16:57 |
| *** alff has joined #openstack-infra | 16:57 | |
| *** alff_ has joined #openstack-infra | 16:57 | |
| fungi | ruhe: right, i think if your project requirements aren't suitable for requirements gating yet, you need to make that job non-voting and also make sure your project name is not present in openstack/requirements:projects.txt on relevant branches | 16:58 |
| *** andreaf has quit IRC | 16:59 | |
| *** ominakov has joined #openstack-infra | 16:59 | |
| fungi | ruhe: otherwise your ci jobs will also be forced to use our pypi mirror, which is only populated with requirements from openstack/requirements:global-requirements.txt | 16:59 |
| fungi | ruhe: you'll see some other projects also have their versions of that job set to voting: false in the jobs list, like http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n470 | 17:01 |
| fungi | pvo: any idea who manages the iopenedthecloud.com site? the ajaxy submit button on it is broken | 17:02 |
| *** wchrisj_ has joined #openstack-infra | 17:03 | |
| *** prad has joined #openstack-infra | 17:04 | |
| *** MarkAtwood has quit IRC | 17:04 | |
| clarkb | fungi: is tripleo-f20 happy yet? reading sb and it looks like it was building a new image last | 17:05 |
| fungi | clarkb: not yet. i blew away the old images, new one built and went into a ready state, debug log continues to show a deficit of 1 and allocation requests for the label, but nodepool doesn't appear to be trying to build nodes from the image | 17:06 |
| fungi | jeblair: now i'm starting to more strongly suspect something's not quite kosher with the allocation logic | 17:06 |
| fungi | i can try bumping the min-ready a little in nodepool.yaml and see what happens | 17:07 |
| fungi | right now it's 1 | 17:07 |
| clarkb | fungi: did pabelanger's changes around min-ready defaults merge? I guess that shouldn't affect this either way since we have an explicit value of 1 | 17:07 |
| fungi | though dsvm-precise-krnl has min-ready: 1 and it builds nodes | 17:08 |
| derekh | fungi: cool, thanks, will be back in a bit to check it out | 17:08 |
| *** derekh has quit IRC | 17:08 | |
| *** MarkAtwood has joined #openstack-infra | 17:08 | |
| ruhe | fungi: thanks for input. but i'm speaking about dvsm jobs. they also fail because of requirements | 17:10 |
| fungi | ruhe: they shouldn't if you take your project out of the projects.txt in openstack/requirements i think | 17:11 |
| ruhe | fungi: here is an example http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n470 | 17:11 |
| ruhe | fungi: here is correct link http://logs.openstack.org/61/85061/1/check/gate-murano-devstack-dsvm/98eb517/logs/devstacklog.txt.gz#_2014-04-03_15_04_23_013 | 17:11 |
| *** unicell has quit IRC | 17:13 | |
| ruhe | fungi: and murano is not present in openstack/requirements | 17:14 |
| *** harlowja_away is now known as harlowja | 17:14 | |
| fungi | ruhe: yeah, i think it's not supposed to be running _sync_requirements_file() | 17:14 |
| fungi | looking at devstack to see why it might | 17:14 |
| *** nicedice has joined #openstack-infra | 17:16 | |
| ruhe | fungi: it seems that we need to use setup_develop_no_requirements_update from devstack/functions-common | 17:17 |
| ruhe | i'll try that | 17:17 |
| *** yjiang has joined #openstack-infra | 17:18 | |
| rcarrillocruz | hmm, got two reviews that failed at job check-tripleo-ironic-undercloud-precise , and i don't think my code has anything with that, | 17:18 |
| rcarrillocruz | anyone else getting that job failing? | 17:18 |
| *** mihgen has quit IRC | 17:19 | |
| clarkb | rcarrillocruz: I think that may be a known problem as tripleo + ironic isn't working yet. #tripleo can confirm | 17:20 |
| rcarrillocruz | thx clarkb, i'm diggint in the console logs, will check it out at #tripleo | 17:21 |
| *** zns has joined #openstack-infra | 17:22 | |
| *** yjiang has quit IRC | 17:23 | |
| fungi | ruhe: the only place in devstack i find a call to setup_package (formerly known as setup_develop_no_requirements_update) is within setup_package_with_req_sync | 17:24 |
| clarkb | fungi: reading https://review.openstack.org/#/c/84828/1 if the project has the overriden branch I would expect that branch to be used | 17:24 |
| fungi | clarkb: even if the change being tested is not on that branch? | 17:24 |
| fungi | clarkb: because under those circumstances, you wind up not testing the change | 17:25 |
| clarkb | fungi: right I don't think this was intended on testing changes but for use in periodic jobs | 17:25 |
| clarkb | I need to go reread job defs really quickly I guess | 17:25 |
| fungi | clarkb: and if the change being tested *is* targeting that branch, then it will be used anyway without needing to override the branch for the zuul_project | 17:25 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: fix gerrit plugin build job https://review.openstack.org/85114 | 17:25 |
| clarkb | actually I think the term 'override' here is confusing me | 17:26 |
| fungi | clarkb: the override is used in periodic bitrot jobs and backwards-compat test jobs presently | 17:26 |
| clarkb | brnach override is for periodic jobs, this is a ref override? | 17:26 |
| fungi | clarkb: for example, if you make a change to devstack-gate master, one variant of jobs tests that change with the override set to stable/havana | 17:26 |
| clarkb | fungi: right, which aren't changes | 17:27 |
| clarkb | fungi: ya, so I think we want the existing behavior | 17:27 |
| fungi | clarkb: which works with the current setup because devstack-gate only has a master branch | 17:27 |
| clarkb | the idea is test everything on a common branch or fall back on master otherwise | 17:27 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: jenkins-job-builder: Allow read/write stdin/stdout https://review.openstack.org/84778 | 17:27 |
| *** mrodden has joined #openstack-infra | 17:27 | |
| fungi | clarkb: but then trying to test master tempest changes against stable/havana projects didn't work the same way, because tempest actually had a stable/havana branch which ended up used as a fallback | 17:28 |
| clarkb | fungi: so, I think you need a new featuer in d-g to support that | 17:28 |
| clarkb | fungi: the existing feature is working as expected | 17:28 |
| *** MarkAtwood has quit IRC | 17:29 | |
| fungi | clarkb: i disagree, because the existing feature has a hole which lets you not test your proposed change if you have a backward-compat job overriding to a branch that project actually has | 17:29 |
| clarkb | fungi: but your change implicitly disables an explicit override | 17:29 |
| fungi | clarkb: take for example devstack-gate... if we suddenly added a stable/havana branch (not that we would be likely to do so) the backward-compat jobs for it would suddenly cease working correctly | 17:30 |
| *** MarkAtwood has joined #openstack-infra | 17:30 | |
| *** MarkAtwood has quit IRC | 17:30 | |
| clarkb | fungi: right, I am not disagreeing we may want to test what you describe. I am saying that implictly breaking an explicit override is the wrong way to do it | 17:30 |
| clarkb | the point of that code is to do what it currently does | 17:30 |
| fungi | clarkb: the override is for all projects, but i think it really should be for all projects if there's no change to be tested, and for all projects except the project which has the change to be tested otherwise | 17:30 |
| *** VijayTripathi has quit IRC | 17:31 | |
| fungi | clarkb: "the point of that code is to do what it currently does" could be used to negate any bug | 17:31 |
| clarkb | fungi: :P seriously though it is an override saying use this branch | 17:31 |
| clarkb | fungi: which is what it does | 17:31 |
| fungi | the override could just as easily mean use this branch on all projects except the project for the change being tested. under what circumstances would we want to avoid testing the change and just check out a branch instead? | 17:32 |
| clarkb | fungi: also, wouldn't it fall back on master anyways because the ref is against the master branch and not the havana branch | 17:32 |
| clarkb | fungi: I am pretty sure it will fallback as the ref won't be available | 17:32 |
| *** Guest78329 has quit IRC | 17:33 | |
| fungi | clarkb: it tries the zuul ref but modifies it to a nonexistent name (by changing the zuul branch to the override branch), fails to find that (because it isn't there) then looks to see if the project has the override branch and if so uses the tip of that | 17:33 |
| *** morganfainberg_Z is now known as morganfainberg | 17:34 | |
| fungi | thus it never falls back to trying the actual proposed change | 17:34 |
| *** maxbit has joined #openstack-infra | 17:34 | |
| fungi | and, i may be missing something, but i'm failing to identify the circumstances under which we'd ever want a change to be tested by a job which just uses the branch instead of the change being proposed | 17:35 |
| clarkb | the zuul ref looks like refs/zuul/master/Z3b0813341d954e1ca84a0986cb6ee4c3 in your proposed scenario. The override would chnage it to refs/zuul/stable/havana/Z3b0813341d954e1ca84a0986cb6ee4c3. it will clone the stable havana one first which will fail (should because the change was against master), then it will fetch the master ref which does exist | 17:36 |
| *** packet has joined #openstack-infra | 17:36 | |
| *** packet has quit IRC | 17:36 | |
| clarkb | fungi: the behavipr you describe should only happen in periodic jobs | 17:36 |
| clarkb | and change should do the right thing | 17:36 |
| fungi | no, between those it checks to see if the project has a stable/havana branch and just checks that out instead if it finds one, then never falls back to the master ref | 17:36 |
| fungi | i have job logs demonstrating this behavior | 17:37 |
| *** gokrokve has quit IRC | 17:37 | |
| clarkb | fungi: ah ok, so that is where the bug is I think | 17:37 |
| fungi | i agree | 17:37 |
| clarkb | fungi: I think we need a second fallback ref | 17:38 |
| clarkb | first try the override, then try original, then try fallback | 17:38 |
| fungi | i'm not sure why what i have there wouldn't work... the ZUUL_REF won't exist on periodic pipeline jobs, will it? and there won't ever be a ZUUL_REF on the override branch for ZUUL_PROJECT | 17:40 |
| clarkb | fungi: its more a matter of confusion, I am really confused why we would override an override | 17:41 |
| fungi | or actually, there might be if there's a change ahead of it in a dependent pipeline, which could then make matters even worse | 17:41 |
| clarkb | fungi: it would be a different zuul ref | 17:41 |
| clarkb | each change has a unique (as provided by uuid4) zuul ref | 17:42 |
| fungi | right, but refs with that uuid on the branches of any changes ahead of it in the pipeline for the same project | 17:42 |
| clarkb | no, it should only be on the one branch | 17:43 |
| clarkb | pretty sure the merge doesn't bother making refs for every branch | 17:43 |
| *** rainya has quit IRC | 17:43 | |
| fungi | i don't think that's the case... otherwise grenade jobs wouldn't dtrt in a dependent pipeline, for example | 17:43 |
| clarkb | only does it for the one associated with the current change | 17:43 |
| *** thedodd has quit IRC | 17:45 | |
| sdague | jeblair: will do. I'm going to try to sort out some of these tempest master patches first | 17:45 |
| *** jlibosva has quit IRC | 17:45 | |
| clarkb | arg thats right http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/merger/merger.py#n269 | 17:45 |
| fungi | let's say you have two changes to nova in the gate, 1 and 2... 1 is a change to stable/havana and 2 is a change to master | 17:45 |
| fungi | you want grenade to test that upgrading from havana with 1 applied to master with 2 applied works | 17:45 |
| *** jlibosva has joined #openstack-infra | 17:46 | |
| fungi | thus the zuul ref for testing 2 includes the same uuid for nova on both stable/havana and master | 17:46 |
| clarkb | ya | 17:46 |
| *** rlandy has quit IRC | 17:46 | |
| sdague | I think I've got a mobius strip of race -1s. A -1ed by bug B, fix for bug -1ed by bug C, fix for C -1ed by bug A. | 17:46 |
| clarkb | sdague: put some hot wheels on it, free energy | 17:47 |
| clarkb | fungi: so, following that train of thought, does this new override behavior impact grenade? | 17:47 |
| sdague | heh | 17:47 |
| clarkb | fungi: eg will it break projects that want to be checked out on an override in order to go to a future branch? | 17:47 |
| fungi | clarkb: i don't believe so, because we don't run grenade with override branches | 17:47 |
| fungi | grenade does it's own branch handling logic | 17:48 |
| *** ildikov_ has quit IRC | 17:48 | |
| fungi | right now the override is only used for periodic bitrot jobs and backwards-compatibility jobs | 17:49 |
| fungi | also i think the unit test failing on 84828 is actually incorrect, but haven't looked at how to correct it yet | 17:49 |
| clarkb | fungi: thats what sparked my concern :) | 17:50 |
| fungi | clarkb: so, an example run which i'm trying to fix with that is http://logs.openstack.org/12/82112/8/check/check-tempest-dsvm-full-havana/2a5debd/ | 17:51 |
| clarkb | right so that passed because it checked out tempest stable/havana | 17:51 |
| fungi | if you look in the workspace setup log you can see where it's selecting stable/havana tip for openstack/tempest rather than the zuul ref | 17:51 |
| clarkb | fungi: I understand the problem we are trying to fix. I am just not convinced that fix is the one we want. Mostly because it is really confusing to override and override | 17:52 |
| sdague | clarkb / fungi: what's the nodepool implications of this - https://review.openstack.org/#/c/85057/ | 17:52 |
| sdague | will I be breaking things badly before the images regen? | 17:52 |
| sdague | I'm going to change the heat guest image we use for testing to be an architecture match with the hosts we're on | 17:53 |
| *** che-arne has quit IRC | 17:53 | |
| sdague | in the hope of being a little faster | 17:53 |
| *** khyati has joined #openstack-infra | 17:53 | |
| fungi | looks like 214mb? | 17:53 |
| clarkb | sdague: every test will have to download that image and can't rely on its cache | 17:54 |
| clarkb | sdague: whcih may cause things to fail if it takes a long time or if the downloads are unreliable | 17:54 |
| fungi | right. some of those jobs may fail if the download is interrupted. also we might ddos the fedora mirrors ;) | 17:55 |
| sdague | clarkb: could we trigger a nodepool rebuild manually post merge? | 17:55 |
| clarkb | sdague: you can do it in two steps, add new image to stackrc, then use new image/remove old image to do it in a safer way | 17:55 |
| clarkb | sdague: we can but some regions aren't reliable in that manner | 17:55 |
| clarkb | sdague: eg it can take days to get a new image | 17:55 |
| sdague | interesting | 17:55 |
| zaro | mgagne1: what's up? new nick now? | 17:55 |
| fungi | right, and just space those out by a day, and then get us to manually update any images which failed to build overnight (hpcloud-az2, i'm looking at you) | 17:55 |
| sdague | well the problem is the current image is pretty problematic I think | 17:56 |
| clarkb | fungi: I am poking at d-g unittests now to try and grok the failure | 17:56 |
| fungi | i have a shell while loop i can use to repeatedly retry updating a nodepool image without deleting all the existing images to get nodepool to do that itself | 17:56 |
| fungi | that way we don't lose test capacity due to unavailable images in a provider | 17:57 |
| sdague | ok | 17:58 |
| *** derekh has joined #openstack-infra | 17:58 | |
| fungi | though in tough times, that loop has had to run for close to 48 hours nonstop to get a fresh image update to complete in hpcloud-az2 because of the current metadata server instability there | 17:59 |
| sdague | ok | 17:59 |
| sdague | interesting | 17:59 |
| clarkb | fungi: I think the indication of that unittest is d-g expects to setup branches for grenade (whether or not that is actually the case is another question) | 18:00 |
| mordred | clarkb: d-g unittests? | 18:00 |
| *** whoops has joined #openstack-infra | 18:00 | |
| jeblair | mordred: i wrote some a while back. :) | 18:00 |
| clarkb | fungi: d-g appears to do that setup for us. it does setup_workspace for old/ and new/ | 18:01 |
| clarkb | fungi: so I think that change does break grenade | 18:01 |
| jeblair | shouldn't it only break grenade if the override is set? | 18:01 |
| clarkb | jeblair: yes | 18:01 |
| clarkb | jeblair: actually wait, maybe not | 18:02 |
| fungi | right, i think grenade and the override branch logic are probably fundamentally incompatible anyway | 18:02 |
| clarkb | jeblair: because grenade is changing $branch to setup_project which is hwo the override works too | 18:02 |
| clarkb | so the override is overloaded | 18:02 |
| jeblair | so we could decide to explicitly not support that case | 18:02 |
| fungi | also a valid option | 18:03 |
| clarkb | ya so grenade would be affected always | 18:03 |
| clarkb | because grenade is effectively a branch override | 18:03 |
| *** rwsu has quit IRC | 18:03 | |
| jeblair | or we can define its behavior as "it overrides what we would normally do with grenade" which is to say, if you set override_tempest_branch to master, then you get tempest master in both grenade checkouts | 18:03 |
| sdague | jeblair: I actually think tempest on both branches is acceptable | 18:04 |
| sdague | sorry, tempest master on both sides | 18:04 |
| sdague | it's actually the behavior we want | 18:04 |
| jeblair | sdague: yeah, it's thinking a few steps ahead here, but that does seem like it might match what you want to do | 18:04 |
| sdague | I also agree, grenade is all kinds of edge cases | 18:04 |
| *** dizquierdo has quit IRC | 18:06 | |
| sdague | clarkb: so yeh... mirror fail isn't theoritical - http://logs.openstack.org/78/85078/2/check/check-tempest-dsvm-full/91f8c9b/logs/devstacklog.txt.gz#_2014-04-03_16_00_13_042 | 18:07 |
| sdague | when will we know that a nodepool has updated images? | 18:08 |
| clarkb | sdague: http://nodepool.openstack.org has logs you can check, or we can do a listing for you | 18:09 |
| fungi | sdague: basically we should just merge the first part on one day, then check in the second day and i can manually update any stragglers which failed to work on the daily periodic updates, and let you know once they're caught up so you can approve the second half | 18:10 |
| sdague | fungi: ok | 18:10 |
| sdague | let me figure out a bridge patch for that | 18:10 |
| *** thedodd has joined #openstack-infra | 18:11 | |
| fungi | or if it's urgent, i can manually update all of them between patches, but would rather not unless you really need it asap | 18:11 |
| sdague | well... here's the challenge, in this new model I was having yuo get this image if you had heat enabled | 18:11 |
| sdague | I guess I can trick nodepool by putting the imageurl add in there in a block devstack never executes? | 18:12 |
| *** gokrokve has joined #openstack-infra | 18:12 | |
| *** malini has joined #openstack-infra | 18:13 | |
| fungi | or break the urls out into a separate file which you reference from the script logic, and we can tweak our prep scripts to grab everything in that file (or everything in a certain section of the file, or with a specific comment tag, or...) | 18:13 |
| fungi | doing something with downloads similar to the way debs and rpms are handled might make devstack a little more extensible anyway | 18:14 |
| sdague | ok, so there is a related impact which is this patch - https://review.openstack.org/#/c/81486/ | 18:15 |
| *** talluri has quit IRC | 18:15 | |
| sdague | will the nodepool parser be able to handle that? | 18:15 |
| sdague | or does that beg the issue of a bash script in devstack to evaluate out all the image urls? | 18:15 |
| fungi | sdague: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/cache_devstack.py#n82 | 18:16 |
| *** phschwartz has quit IRC | 18:17 | |
| fungi | sdague: so only lines in stackrc which start with IMAGE_URLS will match | 18:17 |
| sdague | right, but us moving to vars in urls | 18:17 |
| *** phschwartz has joined #openstack-infra | 18:18 | |
| sdague | would seem to break it | 18:18 |
| fungi | sdague: and that change has them commented, thus #IMAGE_URLS (which doesn't match_ | 18:18 |
| jogo | ahh marconi now has a PTL candidate | 18:18 |
| sdague | fungi: no, that wasn't what I was concerned on | 18:18 |
| fungi | sdague: oh | 18:19 |
| sdague | IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros- | 18:19 |
| sdague | ${CIRROS_VERSION}-x86_64-uec.tar.gz"};; | 18:19 |
| sdague | I don't see how the cache handles the vars | 18:19 |
| fungi | sdague: ahh, right, it doesn't seem to | 18:19 |
| fungi | sdague: it strips away to default variable values if they're specified | 18:20 |
| *** chrisstreeter_ has joined #openstack-infra | 18:20 | |
| fungi | but not recursively | 18:20 |
| sdague | ok, so in the stack of fixes I think we need then, is actually the script in devstack to generate all the image urls | 18:21 |
| *** phschwartz has quit IRC | 18:21 | |
| fungi | and only for the IMAGE_URLS envvar itself | 18:21 |
| mtreinish | jogo: you can still run :) | 18:21 |
| fungi | sdague: agreed. i think we should simplify the parsing in cache_devstack.py by making devstack carry or generate a list of urls | 18:22 |
| zaro | anteaya: have you tried to do the puppet for gerrit? | 18:22 |
| anteaya | no | 18:22 |
| sdague | fungi: ok, before I end up with stack overflow, let me do that one | 18:22 |
| anteaya | i have not tried to use the gerrit puppet module | 18:22 |
| zaro | anteaya: do you have puppet setup in a vm? | 18:22 |
| sdague | then work backwards on these | 18:22 |
| anteaya | I found the config overwhelming | 18:22 |
| *** phschwartz has joined #openstack-infra | 18:22 | |
| *** dkliban has quit IRC | 18:23 | |
| zaro | anteaya: haha!! that's why i haven't done it either. | 18:23 |
| fungi | sdague: and we likely need to phase that too so we can do phased deprecation across an image build (or two) | 18:23 |
| anteaya | zaro: no i have just been working on setting up a gerrit manually and working backward with the puppet manifest | 18:23 |
| anteaya | makes sense to me | 18:23 |
| fungi | sdague: or else temporarily complicate cache_devstack.py by assembling and deduping the list from both sources | 18:24 |
| sdague | yep | 18:24 |
| zaro | who's the person to talk to about puppet dashboard? | 18:25 |
| fungi | anteaya: zaro: fwiw, that's what i did the first time too... made the puppet spaghetti easier to follow once i had done most of the same steps by hand | 18:25 |
| zaro | i want to add review-dev.o.o to it. | 18:25 |
| fungi | zaro: is puppet agent running on review-dev? | 18:25 |
| anteaya | zaro: nibalizer is probably the best person | 18:25 |
| zaro | fungi: probably not. | 18:25 |
| fungi | zaro: it should just automatically show up if so | 18:25 |
| zaro | fungi: not sure what i should do about that. | 18:26 |
| anteaya | zaro: I can keep stepping through it by hand and let you know when I am in a better position for understanding the puppet manifest | 18:26 |
| anteaya | then we can work on seeing how to make it work to install gerrit | 18:26 |
| anteaya | if you want | 18:26 |
| fungi | zaro: all puppetboard is doing is displaying info about reports it finds in puppetdb, which in turn are merely forwarded by the puppetmaster, which it does for any it receives from running puppet agents | 18:26 |
| mgagne1 | Anyone going to pycon? | 18:27 |
| zaro | fungi: i have review-dev setup with gerrit-2.8, but if i let puppet install gerrit i don't think all the patches have merged for it to work. | 18:27 |
| *** mgagne1 is now known as mgagne | 18:27 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gear: Add access control https://review.openstack.org/84873 | 18:27 |
| zaro | fungi: running puppet won't update the server? | 18:27 |
| jeblair | fungi: do i need to look at the nodepool allocator? | 18:27 |
| fungi | zaro: right, basically puppetboard is a display of systems running the puppet agent, and tells you what the agent has done. if the agent isn't running on review-dev, then there's little reason for it to appear on puppetboard anyway right? | 18:27 |
| *** mwagner_lap has quit IRC | 18:27 | |
| fungi | jeblair: yeah, i mentioned it earlier but it probably got buried in the devstack{,-gate} discussion. i'm seeing if i can spot the logic error (assuming there is one to spot) | 18:28 |
| zaro | fungi: agreed. so i would like it to run the agent but that will probably break gerrit on it. | 18:29 |
| zaro | fungi: gerrit won't work until a few more patches for 2.8 is merged. | 18:29 |
| fungi | zaro: unelss the review_dev.pp passes through enough of the changes you've been making locally to reflect what you need | 18:29 |
| fungi | which might not be worth the effort until it's closer to solved | 18:30 |
| *** alexpilotti has joined #openstack-infra | 18:30 | |
| zaro | anteaya: sure. let know if you run into troubles by hand. | 18:30 |
| anteaya | okay thanks | 18:31 |
| anteaya | about every 5 minutes | 18:32 |
| anteaya | I'll take notes | 18:32 |
| zaro | fungi: ok. i think i get it. i'll check all of the changes on my private puppet vm before running the agent on review-dev. | 18:34 |
| zaro | anteaya: really wanted to put this off, but now i really need to run the puppet. | 18:35 |
| *** dmsimard has left #openstack-infra | 18:36 | |
| anteaya | yeah, we are getting close to upgrade | 18:36 |
| anteaya | I don't think running the puppet with be a 2 day thing to figure out how to do | 18:36 |
| zaro | mgagne: no pycon for me. did you want to have an 'add me' to reviewers on gerrit? | 18:37 |
| *** derekh has quit IRC | 18:37 | |
| anteaya | mgagne: mgagne1 yea pleia2 and I are | 18:38 |
| anteaya | I still have to book my train | 18:38 |
| mgagne | zaro: http://i.imgur.com/DncoG3V.jpg | 18:38 |
| anteaya | lol | 18:39 |
| *** derekh has joined #openstack-infra | 18:39 | |
| pleia2 | hehe | 18:40 |
| *** dmsimard has joined #openstack-infra | 18:41 | |
| fungi | jeblair: from the debug log it looks like label_demand.get(label.name, 0) is perpetually 0 for that label | 18:41 |
| fungi | jeblair: however demand is 1, so maybe label_demand[label.name] = demand isn't working? | 18:42 |
| *** dmsimard has left #openstack-infra | 18:43 | |
| fungi | though if not, i would think that would break building nodes for the other labels... :/ | 18:43 |
| zaro | mgagne: your wish is my command.. https://gerrit-review.googlesource.com/#/c/55651 | 18:43 |
| mgagne | zaro: :D | 18:44 |
| mgagne | zaro: they look to not like my idea lol | 18:44 |
| zaro | mgagne: :( | 18:44 |
| mgagne | zaro: greasemonkey then =) | 18:45 |
| pabelanger | fungi, is there anyway to have nodepool not use a saved image, but launch / bootstrap / attach jenkins slave all at once? | 18:46 |
| zaro | mgagne: maybe you can tell them about your use case. | 18:46 |
| mgagne | zaro: well, he proposed a "solution" wish is to -1 the change until you have time to review. | 18:47 |
| mgagne | zaro: but my use case might be limited to OpenStack as AFAIK we have a custom dashboard right? | 18:47 |
| mgagne | zaro: and changes won't show up to this dashboard until I'm added to reviewers | 18:47 |
| jeblair | fungi: okay, i'll look into it after lunch (maybe i can make a test case) | 18:48 |
| zaro | mgagne: custom dashboard? | 18:48 |
| mgagne | zaro: isn't the homepage a dashboard written/coded by OpenStack team? | 18:49 |
| zaro | mgagne: not sure what you mean, gerrit web page is from upstream. | 18:49 |
| *** zns has quit IRC | 18:49 | |
| zaro | mgagne: nope. it's only the 'important changes' page that's custom. | 18:49 |
| mgagne | zaro: oh | 18:49 |
| zaro | mgagne: and that will probably go away after we upgrade gerrit. | 18:49 |
| *** marun is now known as marun_afk | 18:50 | |
| sdague | fungi: how about this - Sean Dague proposed a change to openstack-dev/devstack: add an image_list tool https://review.openstack.org/85142 ? | 18:50 |
| *** msabramo has joined #openstack-infra | 18:51 | |
| sdague | the output is one image per line | 18:51 |
| sdague | full url | 18:51 |
| jeblair | sdague, fungi, clarkb, mordred, SergeyLukjanov: so i heard back from the 1st of the three possible new node providers -- they can offer 100@2cpux8gb or 50@4cpux8gb, or, presumably (since they'd have to add it to their flavor catalog), 25@8cpux8gb | 18:51 |
| jeblair | it seems they are cpu constrained | 18:51 |
| sdague | how good are their cpus? | 18:51 |
| jeblair | they are open to adding more capacity later | 18:51 |
| jeblair | (as they grow their service and add more pods) | 18:51 |
| sdague | 2cpu doesn't do us any real good at this point | 18:52 |
| sdague | 4 is good if the cpus themselves are good | 18:52 |
| sdague | re: newish | 18:52 |
| sdague | 8 is awesomer | 18:52 |
| jeblair | they say they are not oversubscribed | 18:52 |
| clarkb | the only reason we use 8cpu nodes at all is that particular provider limits cpu time in fun ways | 18:52 |
| sdague | yeh, it's not even oversubscribed I think | 18:52 |
| clarkb | ++ to 4cpu | 18:52 |
| sdague | the old rax 4cpu were slow because they were amd chips with low cache | 18:53 |
| *** ominakov has quit IRC | 18:53 | |
| jeblair | sdague: don't we use 8 cpus on the nice new fast rax nodes though? | 18:53 |
| sdague | jeblair: we use 8 cpu on them | 18:53 |
| sdague | core for core they are still slower than hp | 18:53 |
| sdague | well then hp 1.0 | 18:53 |
| *** dteselkin_ has quit IRC | 18:54 | |
| *** derekh has quit IRC | 18:54 | |
| jeblair | okay, so maybe both hp 1.1 and rax are limited in some way | 18:54 |
| sdague | hp 1.1 is very clearly limitted | 18:54 |
| sdague | but I think that's intentional | 18:55 |
| clarkb | ya | 18:55 |
| sdague | honestly, the per core performance on hp 1.0 is somewhat astonishing | 18:55 |
| *** meera has joined #openstack-infra | 18:55 | |
| jeblair | so the worst case is that we end up with 25 nodes to start. we might end up with 50. | 18:55 |
| sdague | it was head to head as good as some very respectable physical hardware I had at IBM | 18:55 |
| sdague | yeh | 18:56 |
| sdague | I'd say lets try at 4 | 18:56 |
| jeblair | and we might end up with more over time | 18:56 |
| sdague | and figure out if it's limitted enough that we really want to switch to 8s instead | 18:56 |
| jeblair | considering the time we'll invest in benchmarking, setting up, etc, is that worth it? | 18:56 |
| sdague | jeblair: fair | 18:56 |
| sdague | sure, I guess everything else is 8s | 18:56 |
| jeblair | sdague: oh no, sorry, i'm fine trying 4... | 18:57 |
| sdague | jeblair: oh, yeh, I think 4s are fine | 18:57 |
| sdague | lets start there | 18:57 |
| jeblair | sdague: i meant should we proceed in general, with either (i mean, we'd still need to figure out what they're doing differently even with 8 -- everyone does something different :( | 18:57 |
| sdague | jeblair: ++ we should proceed | 18:58 |
| mordred | jeblair: well, each time we do that, we learn more and make our overall system more resilient | 18:58 |
| sdague | both for getting more resource for infra | 18:58 |
| mordred | one of these times we'll add a new system and it'll Just Work | 18:58 |
| sdague | and for helping learn how different OS is in the field | 18:58 |
| *** ildikov_ has joined #openstack-infra | 18:58 | |
| sdague | because I think that understanding of differences is very useful feedback to the community, the board, and the vendors | 18:58 |
| nibalizer | zaro: hi, puppetboard.o.o you mean? | 18:59 |
| nibalizer | puppetboard is a unique project from puppet dashborad | 18:59 |
| nibalizer | review-dev doesn't seem to be checking in | 18:59 |
| *** _nadya_ has joined #openstack-infra | 19:00 | |
| nibalizer | which is strange, is that node running puppet? | 19:00 |
| nibalizer | success, or failure, it should be submitting reports to puppetdb | 19:00 |
| nibalizer | i do see a node definition in site.pp | 19:00 |
| jeblair | okay. i think 25 won't make a huge difference for our resource usage, 50 is getting there, 100 is what i was shooting for. but you're right, there are other intagibles, and working to establish a relationship is probably worth it. | 19:00 |
| *** alff_ has quit IRC | 19:01 | |
| *** alff has quit IRC | 19:01 | |
| fungi | pabelanger: no feature for that yet, but probably not too complicated to add if you wanted to hack on it | 19:02 |
| clarkb | did board meeting just start? | 19:03 |
| clarkb | anyone have a number for that? | 19:03 |
| mordred | yah | 19:03 |
| mordred | 1-650-479-3207 | 19:03 |
| mordred | Session number: 925 304 456 | 19:03 |
| fungi | sdague: that looks reasonable (assuming it works--i haven't tested it). as long as we can run that statelessly in a fresh clone of devstack, we should be able to use that to determine what to download and precache | 19:03 |
| jeblair | and #openstack-board | 19:03 |
| pabelanger | fungi, Ya, was considering adding it. Having some problems with saved images launching (delayed) so wanted to see if there was another way | 19:04 |
| sdague | fungi: it works locally | 19:04 |
| pabelanger | but, working on a patch to implement a launchnode timeout, code is there but not exposed in yaml file | 19:04 |
| sdague | it also has a safety latch to exit if < 5 images are returned (the current known sane number) | 19:05 |
| sdague | so check the exit code on it in use | 19:05 |
| pabelanger | we'll wait 3600 mins before cancelling the build | 19:05 |
| pabelanger | Rather way 15mins :) | 19:05 |
| pabelanger | wait( | 19:05 |
| pabelanger | grr | 19:05 |
| *** thedodd has quit IRC | 19:06 | |
| fungi | jeblair: agreed, even if we start out small, the potential to scale that up in the future might be worth the up-front effort as well | 19:06 |
| clarkb | woot finallymanaged to make a meeting, I am putting them on my calendar now | 19:06 |
| *** mwagner_lap has joined #openstack-infra | 19:08 | |
| fungi | was worried i'd dialed in too late, but they're still starting | 19:09 |
| msabramo | very easy PR for jenkins-job-builder - probably ready for merge - https://review.openstack.org/84950; could squeeze it in before meeting starts… :-) | 19:10 |
| *** derekh has joined #openstack-infra | 19:11 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: allow for generic branch overrides https://review.openstack.org/84829 | 19:11 |
| sdague | I finally figured out why the generic branch bit was exploding on grenade | 19:12 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Tightened up story detail UI https://review.openstack.org/84882 | 19:12 |
| sdague | $OVERRIDE_OSLO.VMWARE_BRANCH | 19:13 |
| sdague | not so much a real bash variable | 19:13 |
| jeblair | haha | 19:13 |
| clarkb | fungi: jeblair does https://review.openstack.org/#/c/84829/ fix the d-g thing we were talking about earlier? | 19:14 |
| fungi | clarkb: it does a different thing | 19:14 |
| *** ilives has joined #openstack-infra | 19:14 | |
| fungi | clarkb: that's where we can say "test this nova stable/havana change using tempest master" | 19:15 |
| clarkb | fungi: right, which I thought was relasted to what you were doing and tempest stable/havana was being used instead of master | 19:15 |
| phschwartz | Anyone know the travel time between the conference center where the summit is and Hartford Jackson? | 19:16 |
| dhellmann | fungi: I'm still seeing failures in cliff because of duplicate requirements in master https://jenkins04.openstack.org/job/gate-cliff-requirements/1/console | 19:16 |
| dhellmann | fungi: I thought your change merged? | 19:16 |
| clarkb | dhellmann: we need to wait for new images to be built | 19:16 |
| fungi | clarkb: right, but with that we don't want to override the master branch, while when testing tempest master against stable/havana projects we do | 19:16 |
| dhellmann | clarkb: ah | 19:16 |
| clarkb | dhellmann: since we are image based and not running against static slaves anymore | 19:16 |
| fungi | er, don't want to override the default branch | 19:17 |
| *** chadlung has joined #openstack-infra | 19:17 | |
| dhellmann | ok, the job is non-voting so we'll go ahead | 19:17 |
| dhellmann | clarkb: when do image builds happen? | 19:17 |
| clarkb | fungi: gotcha, master should be used if master change and stable havana if havana change | 19:17 |
| clarkb | dhellmann: early morning EST | 19:17 |
| clarkb | dhellmann: we can also trigger them manually, which we may do if sdague sorts out the fedora image things for heat | 19:18 |
| fungi | clarkb: right, this one just changes the default branch for one project, which doesn't cover the use case of testing tempest master changes against stable server branches | 19:18 |
| dhellmann | clarkb: ok, I'll just ignore that failure for now, since it's only master that has the issue | 19:18 |
| *** thedodd has joined #openstack-infra | 19:21 | |
| *** Clabbe has quit IRC | 19:21 | |
| *** Clabbe has joined #openstack-infra | 19:21 | |
| clarkb | sdague: what does the eval do in that change? it echos the override then evals that? | 19:22 |
| fungi | clarkb: it's so that it can construct the envvars it wants to look for | 19:23 |
| sdague | clarkb: that lets us do the whole project thing dynamically | 19:23 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: ngDelete directive https://review.openstack.org/84504 | 19:23 |
| sdague | instead of having to enumerate them all up front | 19:23 |
| clarkb | fungi: sdague: isn't the echo sufficient? | 19:23 |
| *** jlibosva has quit IRC | 19:23 | |
| sdague | no | 19:23 |
| clarkb | it will echo something or "" | 19:23 |
| sdague | because you have to construct a variable name | 19:24 |
| *** david-lyle has quit IRC | 19:24 | |
| * clarkb learns him a shell | 19:24 | |
| sdague | I'm pretty sure there is no way to do that without the eval, I did try for a while on different approaches | 19:25 |
| sdague | but the problem is you want to check for a variable existing, but you want to build up what that variable name would be from another variable | 19:25 |
| *** zns has joined #openstack-infra | 19:25 | |
| *** malini is now known as malini_afk | 19:26 | |
| *** zns has quit IRC | 19:26 | |
| openstackgerrit | Paul Belanger proposed a change to openstack-infra/nodepool: Create launch-timeout setting for providers https://review.openstack.org/85150 | 19:26 |
| *** jp_at_hp has quit IRC | 19:26 | |
| clarkb | ya, after doing it interactively I see why the eval exists. we need to go from $project_branch_var -> $OVERRIDE_nova_BRANCH -> somebranch | 19:26 |
| sdague | yep | 19:26 |
| *** zns has joined #openstack-infra | 19:27 | |
| *** zns has quit IRC | 19:27 | |
| clarkb | that middle step I didn't grok until just now | 19:27 |
| sdague | yeh, it's a little tricky, and the unit tests are helpful in making sure it's right | 19:27 |
| *** zns has joined #openstack-infra | 19:27 | |
| sdague | I definitely messed it up the first time | 19:27 |
| *** fbo_away is now known as fbo | 19:29 | |
| clarkb | fungi: so run this by me again, this could be used to do tempest master everywhere right? but it can't be used to naturally select the proper branch based on the change branch | 19:30 |
| clarkb | all of this branch stuff is brain melty | 19:30 |
| pabelanger | Hmm | 19:30 |
| fungi | clarkb: right. the two patches are compliments of one another | 19:30 |
| fungi | clarkb: one lets you gate master tempest changes on stable server projects, the other lets you gate stable server changes on master tempest | 19:31 |
| *** alff has joined #openstack-infra | 19:31 | |
| *** alff_ has joined #openstack-infra | 19:31 | |
| *** e0ne has joined #openstack-infra | 19:31 | |
| *** jlibosva has joined #openstack-infra | 19:31 | |
| clarkb | sdague: it is still failing unittests btw | 19:31 |
| sdague | clarkb: really? | 19:32 |
| fungi | clarkb: they're basically the two halves of the symmetry for gating tempest master on stable servers when there's still a stable tempest branch | 19:32 |
| clarkb | ya, see zuul status | 19:32 |
| clarkb | fungi: gotcha thanks | 19:32 |
| sdague | oh, I see | 19:32 |
| pabelanger | well, this is new for nodepool: http://pastebin.com/91S3eAju | 19:33 |
| fungi | clarkb: the fixes/features to support this are potentially useful long-term, even after tempest master gets healthy enough to be able to support stable/havana servers and we subsequently delete the stable/havana branch from tempest | 19:33 |
| pabelanger | just updated to the latest master | 19:33 |
| pabelanger | https://github.com/openstack-infra/nodepool/commit/fca89ee0a0797d7e994c4d0c11c7746f7f136636 seems to be the issue | 19:34 |
| fungi | pabelanger: did zmq get updated? | 19:34 |
| fungi | ooh | 19:34 |
| pabelanger | fungi, let me check my version | 19:34 |
| fungi | pabelanger: i'll get you a pip list from our nodepool server so you can compare versions | 19:35 |
| *** jp_at_hp has joined #openstack-infra | 19:35 | |
| clarkb | it may depend on the C version too which pyzmq via pip should bundle up for you | 19:36 |
| fungi | pabelanger: http://paste.openstack.org/show/75019 | 19:36 |
| *** zns has quit IRC | 19:38 | |
| pabelanger | same version | 19:38 |
| pabelanger | 13.1.0 | 19:38 |
| pabelanger | so, must be build flags | 19:38 |
| clarkb | pabelanger: are you running that on linux? | 19:39 |
| *** zns has joined #openstack-infra | 19:39 | |
| pabelanger | clarkb, ubuntu 12.04 is what I run nodepool on | 19:39 |
| clarkb | wonder if maybe a platform difference | 19:39 |
| sdague | clarkb: ok, one more fix, unit tests running clean again | 19:39 |
| clarkb | pabelanger: thats what we use too thanks | 19:39 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: allow for generic branch overrides https://review.openstack.org/84829 | 19:39 |
| *** thuc has quit IRC | 19:40 | |
| fungi | pabelanger: apparently RCVTIMEO gets passed down into libzmq from what i'm seeing | 19:41 |
| pabelanger | fungi, yes, I think that is my issue | 19:41 |
| pabelanger | I have 2.1.1 installed | 19:41 |
| pabelanger | rebuilding pyzmq now | 19:41 |
| pabelanger | to see if anything shows up | 19:42 |
| *** jlibosva has quit IRC | 19:42 | |
| *** mrda_away is now known as mrda | 19:42 | |
| *** imcsk8_lap has quit IRC | 19:43 | |
| *** thuc_ has joined #openstack-infra | 19:44 | |
| *** imcsk8_lap has joined #openstack-infra | 19:44 | |
| zaro | nibalizer: i meant puppetdb.o.o, i don't think the agent is running on review-dev.o.o | 19:44 |
| *** _nadya_ has quit IRC | 19:44 | |
| *** thuc_ has quit IRC | 19:44 | |
| zaro | nibalizer: i don't want to turn it on yet until i all my patches merge for that server. | 19:44 |
| pabelanger | fungi, are you using a newer version then libzmq1 2.1.11-1ubuntu1 ? | 19:45 |
| *** thuc has joined #openstack-infra | 19:45 | |
| fungi | pabelanger: we don't have libzmq1 installed on our nodepool server | 19:45 |
| clarkb | fungi: pabelanger that is what I expected, pyzmq bundles it into the python lib install if you don't have the system install | 19:45 |
| clarkb | but if system install is there it links against that | 19:45 |
| pabelanger | Ah | 19:46 |
| pabelanger | okay | 19:46 |
| pabelanger | so, that is my issue | 19:46 |
| *** jp_at_hp1 has joined #openstack-infra | 19:47 | |
| sdague | so do we want to merge the layout - https://review.openstack.org/#/c/84864/ | 19:47 |
| sdague | then the d-g job will start generating real results on the branch override? | 19:48 |
| *** markwash has joined #openstack-infra | 19:48 | |
| pabelanger | fungi, clarkb: okay, that was it. Removed the system library | 19:48 |
| *** jp_at_hp has quit IRC | 19:48 | |
| clarkb | sdague: that was my goal, but a second review would be good before I hit the go button | 19:49 |
| *** zhiyan_ is now known as zhiyan | 19:50 | |
| chadlung | Anyone seen this before on DevStack? "tox' is not a global requirement but it should be,something went wrong" | 19:51 |
| clarkb | chadlung: did you add tox to a requirements list somewhere? | 19:52 |
| chadlung | clarkb: Its been in the python-barbicanclient for a while now | 19:52 |
| chadlung | clarkb: just started failing today, did some troubleshooting but can't seem to see where this is happening | 19:53 |
| *** denis_makogon_ has joined #openstack-infra | 19:53 | |
| clarkb | chadlung: was python-barbicanclient recently added to openstack/requirements/projects.txt? | 19:53 |
| clarkb | devstack applies the requirements from global reqs to individual projects to make sure they all work with the global set of reqs. fwiw you rpboably shouldn't hae tox in a dep list | 19:53 |
| clarkb | if not consuming tox directly | 19:53 |
| *** denis_makogon_ is now known as denis_makogon | 19:54 | |
| chadlung | clarkb: We are planning to get rid of it | 19:54 |
| chadlung | clarkb: we don't show up in https://github.com/openstack/requirements/blob/master/projects.txt | 19:54 |
| clarkb | chadlung: ya, I wonder if devstack updated how it applies global reqs | 19:55 |
| *** ilives has quit IRC | 19:55 | |
| *** ilives has joined #openstack-infra | 19:56 | |
| chadlung | clarkb: I was checking commit history, since our devstack gate broke this morning it didn't look like the recent devstack commits would've done that | 19:56 |
| *** rwsu has joined #openstack-infra | 19:56 | |
| fungi | clarkb: i believe this is related to ruhe's issue earlier | 19:57 |
| clarkb | fungi: oh good point /me checks scrollback | 19:57 |
| fungi | i suspect it's a regression in devstack | 19:57 |
| clarkb | ya | 19:57 |
| fungi | clarkb: i can't find anywhere in devstack now that provides a means to conditionally bypass the requirements override and the check for missing global reqs | 19:57 |
| dstufft | chadlung: wat, python-barbicanclient shouldn't have tox in it's requirements, only i's test requirements? unless my understand of pbr or global requirements is wrong :| | 19:58 |
| clarkb | I11acb169e74069be0618e57496ff342f9e788493 that change maybe? | 19:58 |
| clarkb | dstufft: not even its test requirements | 19:58 |
| sdague | what's up? | 19:58 |
| clarkb | dstufft: it is a meta requirement, where if you have it installed you can use it | 19:59 |
| chadlung | dstufft: I'm pulling it out on my fork to test with DevStack. See if that works. | 19:59 |
| sdague | requirements vs. devstack? | 19:59 |
| clarkb | sdague: devstack breaking on global reqs for non integrated things | 19:59 |
| clarkb | sdague: y | 19:59 |
| dstufft | clarkb: chadlung oh well that's probably my fault then | 19:59 |
| dstufft | why it's there that is | 19:59 |
| dstufft | I blame pbr reading requirements.txt | 19:59 |
| dstufft | :D | 19:59 |
| sdague | fungi: jogo added an enforcer on update.py | 20:00 |
| sdague | what we should probably do it make the enforcement optional | 20:01 |
| clarkb | oh it was requirements change that did it? | 20:01 |
| sdague | based on an environment variable | 20:01 |
| sdague | yes | 20:01 |
| sdague | update.py now exits badly if it finds something that is not in the requirements list | 20:01 |
| anteaya | yay | 20:01 |
| fungi | sdague: yep, that seems to be the issue. probably should just dynamically base it on whether the project being setup is in projects.txt in the reqs repo | 20:01 |
| *** bhuvan_ has joined #openstack-infra | 20:01 | |
| anteaya | all programs have at least one ptl candidate | 20:01 |
| anteaya | and as of right now we have 6 elections | 20:02 |
| fungi | anteaya: yep, jogo is out of luck for getting to be a de-facto ptl | 20:02 |
| *** harlowja has quit IRC | 20:02 | |
| sdague | fungi: well, I think that gets more complicated than you imagine | 20:02 |
| anteaya | he will have to be faster next time | 20:02 |
| sdague | I honestly think an override per job that is to ignore non standard requirements would be the right call | 20:02 |
| clarkb | sdague: can't it key off of projects.txt? | 20:03 |
| clarkb | if project not in projects.txt move on | 20:03 |
| sdague | the problem there | 20:03 |
| *** harlowja has joined #openstack-infra | 20:03 | |
| sdague | is if someone gets a git tree into a job, not in projects.txt, it can wedge everything | 20:03 |
| * anteaya goes for a walk | 20:03 | |
| *** ilives has quit IRC | 20:03 | |
| fungi | sdague: clarkb: i suppose devstack-gate could compare ZUUL_PROJECT against openstack/requirements:projects.txt and then export an envvar for whether or not devstack should pass an option into the updater to enforce or not enforce | 20:04 |
| sdague | I think it should be a global lever | 20:04 |
| clarkb | sdague: gotcha | 20:04 |
| clarkb | global per test | 20:04 |
| sdague | clarkb: yes | 20:04 |
| fungi | that, yeah | 20:04 |
| sdague | and not dynamically | 20:04 |
| sdague | so they have to add | 20:04 |
| *** ilives has joined #openstack-infra | 20:04 | |
| sdague | NON_STANDARD_REQS=1 | 20:05 |
| *** bhuvan has quit IRC | 20:05 | |
| sdague | then it would be a warning | 20:05 |
| fungi | so i don't think the job definition needs it, devstack-gate can figure it out, right? | 20:05 |
| *** bhuvan_ is now known as bhuvan | 20:05 | |
| sdague | which would also be really help in understanding which projects are outside of it or not | 20:05 |
| *** _nadya_ has joined #openstack-infra | 20:05 | |
| sdague | fungi: well we got into this because we discovered only very late that trove wasn't in projects.txt or doing req jobs | 20:05 |
| fungi | sdague: otherwise we can no longer reuse any dsvm job names on non-openstack projects | 20:06 |
| sdague | and had a bunch of extra reqs | 20:06 |
| clarkb | fungi: right so sdague is saying it should be explicit and not implicit | 20:06 |
| clarkb | so that stuff doesn't sneak by | 20:06 |
| *** HenryG has quit IRC | 20:06 | |
| *** dprince has quit IRC | 20:06 | |
| sdague | fungi: but we don't anyway right? | 20:06 |
| sdague | because they would co-gate if we did | 20:06 |
| fungi | so we should have one set of dsvm jobs for projects which get enforcement, and another for projects which don't | 20:06 |
| sdague | we are clearly not going to co-gate with a project that doesn't do global reqs | 20:06 |
| fungi | sdague: that's a good point | 20:07 |
| clarkb | ++ | 20:07 |
| *** CaptTofu has joined #openstack-infra | 20:07 | |
| *** rwsu has quit IRC | 20:07 | |
| sdague | ok, so it's late enough in the day that I'm bad at picking variable names | 20:08 |
| sdague | so if someone picks it, I'll do the update.py patch | 20:08 |
| clarkb | fungi: https://review.openstack.org/#/c/84828/ has a comment now. Ithink that will fix the test problem | 20:08 |
| fungi | $FRED={branch:-foo} | 20:08 |
| fungi | even my joke code has syntax errors. oh well | 20:08 |
| *** rwsu has joined #openstack-infra | 20:09 | |
| clarkb | fungi: close enough :) | 20:09 |
| fungi | clarkb: thanks--i'll see if i can get that to pass | 20:09 |
| clarkb | jogo: https://review.openstack.org/#/c/84913/1 wow that is really unhappy :) | 20:10 |
| *** rwsu has quit IRC | 20:11 | |
| *** rwsu has joined #openstack-infra | 20:11 | |
| *** ociuhandu has quit IRC | 20:12 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Increase min-ready for tripleo-f20 nodes https://review.openstack.org/85160 | 20:13 |
| sdague | how do I fixture the environment? | 20:13 |
| clarkb | sdague: one sec | 20:14 |
| clarkb | sdague: https://pypi.python.org/pypi/fixtures EnvironmentVariable ? | 20:14 |
| *** malini_afk is now known as malini | 20:15 | |
| jogo | clarkb: global reqs broke it, have to test locally first | 20:16 |
| clarkb | jogo: because those packages aren't in the mirror? | 20:16 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: New gerrit group - oslo-messaging-core https://review.openstack.org/85162 | 20:17 |
| clarkb | mtreinish: re https://review.openstack.org/#/c/84840/1 I am pretty sure I pointed that out to sdague when it went in and he wanted that behavior is this something we should chat about before fixing? | 20:19 |
| *** julim has quit IRC | 20:19 | |
| * fungi is disappearing for a bit to cook dinner | 20:19 | |
| sdague | clarkb: honestly, I could go either way at this point | 20:19 |
| mtreinish | clarkb: well it doesn't make any sense to run non neutron jobs on neutron patches | 20:19 |
| mtreinish | it just wastes a node | 20:20 |
| mtreinish | and bounces neutron patches for no reason | 20:20 |
| mtreinish | but if there's a counter point I'm overlooking we can look at it | 20:21 |
| jogo | clarkb: yup | 20:21 |
| clarkb | mtreinish: not really, more just sanity check since a month ago we went yes this is wrong but oh well :) | 20:21 |
| sdague | mtreinish: well if there is going to be a separate template, it should pull in all the neutron common jobs | 20:22 |
| sdague | and do neutron client as well, right? | 20:22 |
| nibalizer | zaro: puppetboard.o.o is a cname for puppetdb.o.o so we're talking about the same thing | 20:22 |
| nibalizer | zaro: what is your desired end state? | 20:22 |
| nibalizer | basically you won't have review-dev appearing as a node in puppetboard until you run the agent on it | 20:22 |
| sdague | the reason for the one template was most to avoid human error, even if it means there is whitespace where the test doesn't mean anything | 20:22 |
| openstackgerrit | A change was merged to openstack-infra/config: Add stable filters for neutron functional+api jobs https://review.openstack.org/84815 | 20:23 |
| mtreinish | sdague: yeah I should switch neutronclient too | 20:23 |
| mtreinish | one sec I'll update the patch | 20:23 |
| clarkb | sdague: mtreinish: yes covering the client too would be good | 20:23 |
| clarkb | mtreinish: thanks | 20:24 |
| msabramo | clarkb: thanks for the review on https://review.openstack.org/84950 ! | 20:24 |
| sdague | and if it's neutron specific | 20:24 |
| sdague | pull *all* the common jobs between them into that template | 20:24 |
| sdague | so we don't run into the issue again later with only updating one or the other | 20:24 |
| mtreinish | sdague: wouldn't that mean adding the neutron template to every project that uses the integrated-gate template? | 20:26 |
| sdague | mtreinish: no | 20:26 |
| sdague | why would it ? | 20:26 |
| mtreinish | maybe I misread | 20:26 |
| clarkb | I think sdague is proposing two common lists | 20:26 |
| clarkb | one for not neutron and another for neutron | 20:26 |
| clarkb | they may have some overlap, but will be different | 20:27 |
| sdague | right, that | 20:27 |
| *** rwsu has quit IRC | 20:27 | |
| sdague | which is actually what https://review.openstack.org/#/c/84840/ is | 20:27 |
| mtreinish | ok that's probably why I was confused | 20:28 |
| sdague | but if we are bothering to carve off the integrated-gate-neutron template, and it's really only 2 jobs, we should do all the overlap | 20:28 |
| sdague | so add check-grenade-dsvm-neutron, check-tempest-dsvm-neutron-pg | 20:28 |
| sdague | and everything else that's common between neutron and python-neutronclient | 20:29 |
| clarkb | SlickNik: mtreinish sdague https://review.openstack.org/#/c/84369/1 removes a trove job, we hae that coverred as the integrated gate now? | 20:29 |
| sdague | clarkb: yes | 20:29 |
| sdague | trove is in the regular gate now | 20:29 |
| clarkb | woot /me approves | 20:29 |
| sdague | let me double check | 20:29 |
| sdague | http://logs.openstack.org/42/85142/1/check/check-tempest-dsvm-full/0e1d2c9/console.html#_2014-04-03_19_36_48_522 | 20:30 |
| sdague | yes | 20:30 |
| sdague | in the regular gate | 20:30 |
| clarkb | approved | 20:30 |
| *** rwsu has joined #openstack-infra | 20:31 | |
| *** slagle has joined #openstack-infra | 20:31 | |
| *** VijayTripathi has joined #openstack-infra | 20:31 | |
| mtreinish | sdague: does the cells job run with neutron? | 20:33 |
| clarkb | mtreinish: I think no, unless there is a specific cells job for it | 20:33 |
| mtreinish | clarkb: ok then I should remove that from neutronclient too | 20:34 |
| clarkb | mtreinish: let me double check | 20:34 |
| sdague | mtreinish: I doubt it | 20:34 |
| slagle | hi, i don't seem to have permissions to create branches for the tripleo projects (i'm in the tripleo-ptl) group in gerrit | 20:34 |
| slagle | are there some additional permissions that are needed? | 20:34 |
| clarkb | mtreinish: no neutron | 20:35 |
| mtreinish | clarkb: ok thanks | 20:35 |
| clarkb | slagle: which project? I can take a look | 20:35 |
| *** VijayTripathi has quit IRC | 20:35 | |
| clarkb | slagle: any specific examples? | 20:35 |
| slagle | clarkb: tripleo-image-elements is one | 20:35 |
| slagle | clarkb: using the ui, https://review.openstack.org/#/admin/projects/openstack/tripleo-image-elements,branches | 20:35 |
| *** derekh has quit IRC | 20:35 | |
| slagle | if i try to create one called stable/icehouse, i get: Cannot create refs/heads/stable/icehouse | 20:36 |
| clarkb | slagle: so if we look at https://review.openstack.org/#/admin/projects/openstack/tripleo-image-elements,access tripleo-ptl has no refs/heads/* create permissions. Let me get an example of what perms are needed and you can propose that change | 20:36 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Only run jobs with neutron on neutron changes https://review.openstack.org/84840 | 20:36 |
| mtreinish | clarkb, sdague: I think that got everything | 20:36 |
| slagle | clarkb: awesome, thanks | 20:36 |
| *** thuc has quit IRC | 20:37 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove experimental trove devstack-gate job https://review.openstack.org/84369 | 20:37 |
| *** thuc has joined #openstack-infra | 20:37 | |
| *** gondoi is now known as zz_gondoi | 20:39 | |
| *** thuc_ has joined #openstack-infra | 20:39 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Change names for duplicate neutron jobs https://review.openstack.org/83172 | 20:39 |
| clarkb | slagle: actually I can't fine a good example quickly. but under the refs/heads/* perms for triple* files in https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/gerrit/acls/openstack you want to add 'create = tripleo-ptl' | 20:39 |
| clarkb | slagle: that will allow you to create branches | 20:39 |
| clarkb | make those changes, propose them to gerrit and we should get that in for you torrow | 20:40 |
| clarkb | *tomorrow | 20:40 |
| *** khyati has quit IRC | 20:40 | |
| *** david-lyle has joined #openstack-infra | 20:40 | |
| *** thuc__ has joined #openstack-infra | 20:40 | |
| *** rainya- has joined #openstack-infra | 20:40 | |
| *** khyati has joined #openstack-infra | 20:40 | |
| SlickNik | clarkb: As Sean said, it's part of the integrated gate now, so needed to be removed. Thanks! | 20:41 |
| *** thuc has quit IRC | 20:41 | |
| slagle | clarkb: cool, thanks | 20:42 |
| *** _nadya_ has quit IRC | 20:42 | |
| meera | hello all: I'm trying to figure out who I should contact to get resolution on an issue I'm having while doing a git review. I'm trying to submit code to tempest. The issue I'm having is "fatal: ICLA contributor agreement requires current contact information.". I have updated the contact information and in the Agreements settings I have verified the ICLA license agreement exists. I have uploaded newly generated ssh public key to gerrit and | 20:42 |
| meera | launchpad systems. Is there someone who I can work with one on one on getting this issue resolved? | 20:42 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: allow for NON_STANDARD_REQS override https://review.openstack.org/85167 | 20:42 |
| sdague | clarkb: ^^^ | 20:42 |
| sdague | going for a walk while there is sun, but that should be the fix needed for barbican | 20:43 |
| clarkb | mtreinish: reviewed, there is a tyop | 20:43 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Change names for duplicate neutron jobs https://review.openstack.org/83172 | 20:43 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Only run jobs with neutron on neutron changes https://review.openstack.org/84840 | 20:43 |
| clarkb | meera: yes, I can help with that | 20:43 |
| mtreinish | clarkb: yeah, '=' != '-' | 20:43 |
| meera | thanks clark | 20:44 |
| clarkb | meera: are you sure that the usernames match up? make sure the username at https://review.openstack.org/#/settings/ matches the one in the gerrit remote shown by git remote -v | 20:44 |
| clarkb | meera: you may be trying to use two different accounts and the wires are getting crossed | 20:44 |
| *** thuc_ has quit IRC | 20:44 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/config: heatclient: make the py33 gate voting https://review.openstack.org/85170 | 20:45 |
| *** marun_afk is now known as marun | 20:45 | |
| openstackgerrit | James Slagle proposed a change to openstack-infra/config: Allow tripleo-ptl group to create branches https://review.openstack.org/85171 | 20:46 |
| meera | clarkb: THe username match | 20:46 |
| *** dcramer_ has quit IRC | 20:46 | |
| clarkb | meera: can you give me your account number? it is shown on that settings page I linked you | 20:46 |
| clarkb | I will poke at the DB and see what may be going on | 20:46 |
| meera | clarkb: 2846 | 20:47 |
| *** chadlung has quit IRC | 20:49 | |
| clarkb | meera: the db shows you haven't filed contact info. can you update your info at https://review.openstack.org/#/settings/contact and see if that helps | 20:50 |
| meera | clarkb: I have updated the contact info couple of times. I can give it another try | 20:51 |
| *** bhuvan_ has joined #openstack-infra | 20:51 | |
| *** dkehn has quit IRC | 20:51 | |
| clarkb | meera: it isn't showing as having been done. are the changes being saved? | 20:52 |
| *** dkehn has joined #openstack-infra | 20:53 | |
| *** bhuvan has quit IRC | 20:53 | |
| jeblair | lastlog mordred | 20:54 |
| *** bhuvan_ is now known as bhuvan | 20:54 | |
| mordred | lastlog jeblair | 20:54 |
| jeblair | oops; that was my way of looking up the board number again :) | 20:54 |
| mordred | :) | 20:54 |
| meera | clarkb: The updates I made to the Contact information did not include Fax Number. This time around I did put in Fax as well and it went through | 20:54 |
| *** amcrn has joined #openstack-infra | 20:55 | |
| clarkb | meera: I see the DB updated with a timestamps saying they were filed | 20:55 |
| clarkb | meera: try pushing changes now | 20:55 |
| jeblair | but it had the side benefit of showing me: 05:03 * mordred throws wet cats into the air | 20:55 |
| meera | clarkb: it worked | 20:55 |
| meera | clarkb: thanks a lot | 20:55 |
| mtreinish | mordred: could I get added to the cookiecutter-core group after: https://review.openstack.org/#/c/83192/ goes through? | 20:55 |
| mordred | jeblair: you shold never miss such things | 20:56 |
| mordred | mtreinish: sure | 20:56 |
| mtreinish | cool | 20:56 |
| mordred | I beleive I'm one of the only person who ever reviews cookiecutter patches :) | 20:56 |
| *** pfallenop has quit IRC | 20:56 | |
| *** CaptTofu has quit IRC | 20:57 | |
| mtreinish | mordred: well I'll start doing that too then | 20:57 |
| mtreinish | I actually have been meaning to push a couple of updates to the cookiecutter repo that I found making the specs repo | 20:57 |
| mtreinish | I'll try to do that tomorrow | 20:58 |
| *** CaptTofu has joined #openstack-infra | 20:58 | |
| fungi | okay, back... | 20:58 |
| *** chadlung has joined #openstack-infra | 20:58 | |
| mtreinish | clarkb: jenkins likes the latestst revision without the typo :) | 20:58 |
| *** gokrokve_ has joined #openstack-infra | 20:59 | |
| *** lcostantino has quit IRC | 21:01 | |
| *** yolanda has quit IRC | 21:01 | |
| *** CaptTofu has quit IRC | 21:02 | |
| *** pdmars has quit IRC | 21:02 | |
| *** gokrokve has quit IRC | 21:02 | |
| chadlung | Barbican was incubated last month, do we need to add ourselves to openstack/requirements/projects.txt Or...? | 21:03 |
| *** rfolco has quit IRC | 21:03 | |
| fungi | meera: clarkb: just for clarification, https://wiki.openstack.org/wiki/CLA-FAQ#Why_am_I_getting_an_error_message_about_the_ICLA_requiring_contact_information_when_trying_to_submit_patches_for_review.3F | 21:04 |
| clarkb | chadlung: eventually you will need to, first step is getting your reqs to make the global reqs | 21:04 |
| *** aysyd has quit IRC | 21:04 | |
| *** markmc has quit IRC | 21:04 | |
| chadlung | clarkb: cool, thanks | 21:04 |
| clarkb | mtreinish: roger, will rereviews shortly. I need food now that board meeting is over | 21:04 |
| *** dkranz has quit IRC | 21:10 | |
| adam_g | 'OpenStack 2013.2.3 released' needs moderator approval if anyone here has access to approve mail to openstack-announce | 21:11 |
| openstackgerrit | Alex Conrad proposed a change to openstack-infra/jenkins-job-builder: add ability to parse from a file object https://review.openstack.org/85177 | 21:13 |
| fungi | adam_g: i'll check | 21:14 |
| *** ameade has quit IRC | 21:15 | |
| fungi | adam_g: you want ttx, reed or markmc | 21:15 |
| *** dkehn has quit IRC | 21:16 | |
| adam_g | fungi, ah, thanks anyway! | 21:16 |
| fungi | adam_g: you might also want to ask ttx to set you to automatically approved bypassing moderation on that list since you need to send release announcements regularly | 21:16 |
| adam_g | fungi, i think i had that for my old email | 21:17 |
| *** dizquierdo has joined #openstack-infra | 21:17 | |
| fungi | adam_g: right, maybe you just need to update your old subscription? i can't remember if the whitelists follow subscription changes | 21:18 |
| *** dkehn has joined #openstack-infra | 21:18 | |
| *** weshay has quit IRC | 21:19 | |
| *** rwsu has quit IRC | 21:22 | |
| fungi | jeblair: not sure if you did anything, but... | 21:23 |
| fungi | | 3428217 | rax-dfw | bare-precise | jenkins01 | bare-precise-rax-dfw-3428217.slave.openstack.org | bare-precise-rax-dfw-3428217 | 9503f200-3fa4-4cfd-a5b9-733e62cec8f8 | 67.192.246.115 | ready | 0.16 | | 21:23 |
| fungi | jeblair: oh, nevermind | 21:23 |
| lifeless | ok theres something wrong again | 21:23 |
| lifeless | we're seeing jobs try and requeue | 21:24 |
| fungi | i should look closer when i search for three valid hex digits in lists which contain long hex strings | 21:24 |
| fungi | lifeless: i'll take a look in the log | 21:24 |
| *** yassine has joined #openstack-infra | 21:25 | |
| *** dkehn has quit IRC | 21:25 | |
| *** pfallenop has joined #openstack-infra | 21:25 | |
| zaro | nibalizer: yeah, i get it. fungi help me understand that. i'm still hacking on review-dev will run the agent when i'm done with it. | 21:25 |
| fungi | lifeless: nodepool's getting a lot of "Exception: Timeout waiting for ssh access" trying to build new nodes, so i suspect you may have something also killing network connectivity between jenkins and the slave agents as well. looking for stronger evidence | 21:26 |
| *** dkehn has joined #openstack-infra | 21:26 | |
| *** yassine has quit IRC | 21:27 | |
| fungi | also nodepool build errors like "Exception: Timeout waiting for server 856ad456-7e20-4a8c-9c73-49a8799f7955 deletion in tripleo-test-cloud" | 21:27 |
| *** yassine has joined #openstack-infra | 21:27 | |
| lifeless | nova floating-ip-list | 21:27 |
| lifeless | ERROR: HTTPSConnectionPool(host='ci-overcloud.tripleo.org', port=13000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 110] Connection timed out) | 21:27 |
| lifeless | control plane is looking bad | 21:28 |
| fungi | er, delete errors, not build errors | 21:28 |
| mattoliverau | Momring | 21:28 |
| anteaya | hey mattoliverau | 21:28 |
| clarkb | mattoliverau: good momring | 21:28 |
| sdague | wow, fedora mirrors are kind of terribad on availability | 21:28 |
| lifeless | SpamapS: so | 21:28 |
| nibalizer | zaro: cool | 21:28 |
| sdague | I managed to race crash every other devstack change with a fedora mirror timeout on that iso | 21:29 |
| adam_g | is removing a -2 with a 0 expected to re-trigger gate checks? | 21:29 |
| fungi | sdague: no joke. if someone fails to download a cd image, they'll just retry. no biggie right? | 21:29 |
| mattoliverau | anteaya: Tent on a barge huh, sure why not :) | 21:29 |
| sdague | adam_g: if it's old enough, yes | 21:29 |
| anteaya | mattoliverau: wfm | 21:29 |
| anteaya | actually I really like the idea | 21:29 |
| clarkb | sdague: nice | 21:29 |
| sdague | fungi: so that's probably at minimum a 5% fail rate by fedora mirrors | 21:29 |
| sdague | howmuch retry logic is in nodepool? | 21:30 |
| fungi | sdague: i believe none. maybe less | 21:30 |
| sdague | heh | 21:30 |
| fungi | sdague: unless you could trying again the next day as retry logic | 21:30 |
| *** yassine has quit IRC | 21:30 | |
| fungi | s/could/count/ | 21:30 |
| adam_g | sdague, hmph okay. ive got ~70 -2's to remove. :| | 21:30 |
| *** malini is now known as malini_afk | 21:30 | |
| * adam_g thinks we need a better way to handle freezing stable branches in preparation for release | 21:31 | |
| fungi | adam_g: you could script that--gerrit has an api | 21:31 |
| *** yassine has joined #openstack-infra | 21:31 | |
| *** yassine has quit IRC | 21:31 | |
| *** zns has quit IRC | 21:31 | |
| sdague | adam_g: so why not take the time to do actual review on those patches in the process? | 21:31 |
| adam_g | fungi, yeah, i have a script to do it.. but was worried about flooding | 21:31 |
| fungi | adam_g: we'll send agents to your house if the flood gets too bad--that's all | 21:32 |
| sdague | the freshness checks are there for a reason, stable was especially vulnerable to having out of date test results | 21:32 |
| sdague | and wrecking the gate | 21:32 |
| lifeless | [117125.821389] hpsa 0000:06:00.0: cmd_alloc returned NULL! | 21:32 |
| lifeless | [118130.535271] hpsa 0000:06:00.0: cmd_alloc returned NULL! | 21:32 |
| lifeless | rebooting the undercloud | 21:32 |
| adam_g | sdague, many of them are already reviewed. the -2 is just there to prevent htem from merging during the 1 week freeze | 21:32 |
| fungi | adam_g: i wouldn't sweat changing votes on 70 changes in one shot through the api | 21:32 |
| lifeless | erm | 21:32 |
| lifeless | control node | 21:32 |
| sdague | adam_g: sure | 21:32 |
| adam_g | fungi, ack | 21:32 |
| sdague | but they need good test results | 21:32 |
| mordred | clarkb: what's the deal with the new zuul noop stuff? | 21:33 |
| sdague | because if they sat for a week, they might be failing due to a client or library update | 21:33 |
| lifeless | SpamapS: help me rhonda.... | 21:33 |
| clarkb | mordred: noop is a built in job that zuul succeed immedaitely without queueing to gearman | 21:33 |
| adam_g | sdague, i'm not arguing against the re-checks | 21:33 |
| clarkb | mordred: so you can save resources running noop jobs | 21:33 |
| mordred | clarkb: I was just telling cody-somerville about it - but I realized I didn't know that much | 21:33 |
| sdague | so it will just take time | 21:33 |
| *** ociuhandu has joined #openstack-infra | 21:33 | |
| *** dkehn has quit IRC | 21:33 | |
| sdague | but it is actually time well spent | 21:33 |
| mordred | clarkb: so one just s/gate-noop/noop/ in layout.yaml ? | 21:33 |
| adam_g | sdague, thats fine. was just covering my ass in case someone asked 'wtf?' :) | 21:34 |
| fungi | mordred: yep | 21:34 |
| clarkb | mordred: yes, but you need to have a version of zuul new enough to do that | 21:34 |
| mordred | clarkb: kk | 21:34 |
| mordred | cody-somerville: ^^ | 21:34 |
| clarkb | mordred: also my remote fetch change is getting close to being mergable I think | 21:34 |
| mordred | woot | 21:34 |
| clarkb | mordred: so hopefully cody can go to using upstream master soon | 21:34 |
| sdague | and zuul is basically empty | 21:34 |
| mordred | I support such things | 21:34 |
| *** mfer has quit IRC | 21:35 | |
| *** dkehn has joined #openstack-infra | 21:36 | |
| sdague | so a flood will flush in a couple hours | 21:36 |
| sdague | so like fungi said, just bring it on :) | 21:36 |
| sdague | heh | 21:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Add a remote url override location https://review.openstack.org/76057 | 21:36 |
| clarkb | mordred: ^ | 21:36 |
| *** weshay has joined #openstack-infra | 21:37 | |
| clarkb | jeblair: ok now I think that is really ready for proper review. jhesketh has done a good job pushing me along and I think I have addressed his concerns now | 21:37 |
| sdague | adam_g: zuul was getting bored with the light load anyway, time to make him work | 21:37 |
| NobodyCam | any one seen this before? hudson.util.IOException2: remote file operation failed: /tmp/hudson5356793296994292431.sh at hudson.remoting.Channel@179cda28:tripleo-precise-tripleo-test-cloud-3427617 | 21:38 |
| *** zns has joined #openstack-infra | 21:39 | |
| *** HenryG has joined #openstack-infra | 21:39 | |
| *** zehicle has joined #openstack-infra | 21:39 | |
| jeblair | fungi: okay, i am just now starting to look into the allocator | 21:39 |
| *** eharney has quit IRC | 21:40 | |
| *** jhesketh_ has joined #openstack-infra | 21:40 | |
| *** jhesketh_ is now known as jhesketh | 21:41 | |
| fungi | jeblair: yeah, i'm not spotting what's holding the demand down to 0 there | 21:41 |
| jhesketh | Morning | 21:41 |
| fungi | jeblair: the logic in that part of the source looks sound to me | 21:41 |
| clarkb | jhesketh: hi there, thank you for the review on 76057, I think I have it cleaned up now | 21:42 |
| *** jeckersb is now known as jeckersb_gone | 21:42 | |
| jhesketh | cool, I'll take a look | 21:42 |
| clarkb | fungi: derekh was theorizing that the nodes may not be building due to provider manager queueing | 21:42 |
| *** dcramer_ has joined #openstack-infra | 21:42 | |
| clarkb | fungi: bascically they got stuck behind the mass of tripleo-precise stuff (but that doesn't seem super likely0 | 21:42 |
| jeblair | 2014-04-03 21:41:13,126 DEBUG nodepool.NodePool: <AllocationRequest for 1.0 of tripleo-f20> | 21:43 |
| jeblair | 2014-04-03 21:41:13,126 DEBUG nodepool.NodePool: <AllocationSubRequest for 0.0 (out of 1.0) of tripleo-f20 from tripleo-test-cloud> | 21:43 |
| jeblair | clarkb: i think that doesn't support that idea ^ | 21:43 |
| clarkb | jeblair: ya | 21:43 |
| fungi | agreed | 21:43 |
| jeblair | jhesketh: i reviewed up to https://review.openstack.org/#/c/84314/ and ran into a problem | 21:44 |
| jeblair | jhesketh: take a look and let me know what you think. | 21:44 |
| clarkb | jhesketh: one of my inline comment responses about config being rewritten to disk is wrong. If you think my config changes should go in the file and not in the test I can update that too. I personally find it a bit more clear that this new test is doing this additional scenario when it is in the test class | 21:44 |
| *** mbacchi has quit IRC | 21:45 | |
| jhesketh | jeblair: thanks will do | 21:45 |
| jeblair | jhesketh: for what it's worth, and in case it helps your thinking on the subject, i would seriously entertain removing bootstrap. | 21:46 |
| NobodyCam | -infra question? check-tripleo-ironic-undercloud-precise #180 from https://jenkins04.openstack.org/job/check-tripleo-ironic-undercloud-precise/180/ failed with the hudson error above and ended up with a NONE state. is there a way to get the /var/log directory from the vm in this state? | 21:47 |
| jeblair | jhesketh: also, i hope when you say "Morning" what you really mean is "everybody start telling me what they've been doing for the past few hours all at once!" :) | 21:47 |
| clarkb | NobodyCam: see scrollback, the test cloud went sideways | 21:48 |
| clarkb | NobodyCam: I think it is just a side effect of that | 21:48 |
| fungi | NobodyCam: ENOCLOUD | 21:48 |
| jhesketh | jeblair: heh, yep! sure beats scrollback | 21:48 |
| jhesketh | clarkb: sure, I think your fixes are good | 21:48 |
| *** nkinder has quit IRC | 21:48 | |
| NobodyCam | ahhh so recheck it is.. but I'll give it a bit | 21:48 |
| * jeblair goes back to figuring out why 1==0 | 21:48 | |
| *** fbo is now known as fbo_away | 21:48 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added global CSS class to allow style targeting. https://review.openstack.org/85185 | 21:49 |
| jhesketh | clarkb: I don't mind if the change isn't in the CONFIG file but I'm not sure if it has any purpose otherwise | 21:49 |
| jhesketh | I'll do a review later | 21:49 |
| fungi | jeblair: next you'll go on to prove that black is white and get yourself run over at the next zebra crossing | 21:49 |
| *** alexpilotti has quit IRC | 21:49 | |
| NobodyCam | fungi: http://www.math.hmc.edu/funfacts/ffiles/10001.1-8.shtml | 21:49 |
| jhesketh | jeblair: So I'm not very keen on spending massive amounts of time on the zuul status stuff (eg removing bootstrap) | 21:49 |
| clarkb | jhesketh: ya, I think I was trying to demo too much of what I was thinking of in the general case | 21:50 |
| jeblair | jhesketh: heh, that's the same reason i mentioned removing bootstrap. :) | 21:50 |
| clarkb | jhesketh: different base config, different knobs tweaked and so on | 21:50 |
| fungi | NobodyCam: nice! i'd never seen that one | 21:50 |
| NobodyCam | :-p | 21:50 |
| clarkb | jhesketh: http://git.openstack.org/cgit/openstack-infra/zuul/tree/tests/test_scheduler.py#n826 is the line that originally made me think I needed a different file but now I see it is writing to memory and not disk | 21:51 |
| jhesketh | jeblair: heh | 21:51 |
| jhesketh | jeblair: Thanks for the improvements to it though! :-) | 21:51 |
| clarkb | fungi: maybe we can convince jeblair to solve P=NP | 21:52 |
| jhesketh | clarkb: righto, did you want to tidy that up before I re-review? | 21:52 |
| cody-somerville | clarkb: How do you guys deal with nodepool being rate limited? | 21:52 |
| fungi | clarkb: i suspect he already has and is just waiting for the right time to make it public | 21:52 |
| jeblair | fungi: as soon as i'm done with reviews | 21:52 |
| fungi | hah | 21:52 |
| clarkb | jhesketh: sure, do you have a preference between only setting it in memory or setting poll=True on disk and the fetch_url in memory | 21:53 |
| clarkb | cody-somerville: you can set a query rate in the nodepool yaml file | 21:53 |
| *** masayukig has joined #openstack-infra | 21:53 | |
| clarkb | is it in hertz? I would have to reread docs/code to know for sure | 21:53 |
| mordred | clarkb: it should be in hertz if it's not | 21:54 |
| jhesketh | clarkb: Not particularly... I think I'd lean towards config since you have the code there for it, but I do agree with your point about making it slightly clearer what the test is doing differently | 21:54 |
| jeblair | clarkb: i think it ended up being something like 1/hertz | 21:54 |
| jeblair | which should probably be changed | 21:54 |
| mordred | jeblair: perhaps we could start describing nodepool musically | 21:55 |
| clarkb | jhesketh: I am leaning towards removing the on disk file and doing it all in memeory | 21:55 |
| jhesketh | sure | 21:55 |
| *** thuc__ has quit IRC | 21:55 | |
| *** thuc has joined #openstack-infra | 21:56 | |
| *** slagle has quit IRC | 21:57 | |
| jeblair | fungi: has the tripleo backlog ever cleared? | 21:57 |
| fungi | jeblair: no, it made up a lot of ground, but now has begun to grow again due to the latest outage | 21:57 |
| jeblair | i'm wondering if the problem is that nodpool needs 161 servers from tripleo, which can only provide 140 | 21:58 |
| fungi | at one point it was down to ~3 hours behind | 21:58 |
| jeblair | er 40 | 21:58 |
| *** slagle has joined #openstack-infra | 21:58 | |
| jeblair | the f20 demand is 0.6% of the total demand | 21:59 |
| fungi | oh! so basically the demand on the tripleo-precise nodes is flooding out the min-ready on the tripleo-f20 node? | 21:59 |
| jeblair | that's what i'm thinking... will try to confirm | 21:59 |
| pleia2 | there's a patch up to bump min ready to 2 to try to help this | 21:59 |
| fungi | right, if we bumped it to be 4 or 5 i bet that would be enough to tip the scales | 21:59 |
| pleia2 | https://review.openstack.org/#/c/85160/ | 22:00 |
| fungi | since right now the demand is just over 4x the max-servers | 22:00 |
| *** thuc has quit IRC | 22:00 | |
| *** dkranz has joined #openstack-infra | 22:00 | |
| fungi | and 5 nodes min-ready would be enough to get it to garner 1:40 of the max-servers ratio | 22:01 |
| clarkb | jhesketh: gah, as I start doing this I see other potential problems :) | 22:01 |
| clarkb | jhesketh: CONFIG is global so all these sets potentially get shared across tests... | 22:01 |
| fungi | suggesting that maybe we're not rounding up somewhere we ought to | 22:01 |
| jhesketh | heh :-) | 22:01 |
| clarkb | jhesketh: I need to fix this more problem | 22:01 |
| clarkb | s/problem/properly/ | 22:01 |
| fungi | moar problem | 22:01 |
| clarkb | jhesketh: I think it is safe within each test class, but I need a module level global for each of them | 22:02 |
| *** rcleere has quit IRC | 22:03 | |
| *** marun is now known as marun_afk | 22:03 | |
| *** thuc has joined #openstack-infra | 22:04 | |
| *** thomasem has quit IRC | 22:05 | |
| *** zhiyan is now known as zhiyan_ | 22:08 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Add a remote url override location https://review.openstack.org/76057 | 22:09 |
| clarkb | jhesketh: ^ that is sort of an acceptable state. I don't really feel like refactoring the fixtures in this particular change | 22:10 |
| clarkb | jhesketh: but it presents a safe config to each test without cross contamination between classes | 22:10 |
| *** reed has joined #openstack-infra | 22:11 | |
| jhesketh | clarkb: so if you're setting the config as you need in setUp, why do you need the config there at all? | 22:12 |
| jhesketh | oh, it's not a new file | 22:12 |
| clarkb | jhesketh: to isolate that TEST_CONFIG from the TEST_CONFIG used in other tests | 22:12 |
| clarkb | jhesketh: basically we need two sets in memory and that is a cheap way to do it. Should probably be reworked to haev a configFixture that sets things properly for each test but that is a different change | 22:13 |
| *** reed has quit IRC | 22:13 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Stop using Jenkins SCM plug-in https://review.openstack.org/81302 | 22:13 |
| *** HenryG_ has joined #openstack-infra | 22:14 | |
| jhesketh | clarkb: why not just move the ConfigParser into SetUp? | 22:14 |
| jhesketh | (ie move CONFIG) | 22:15 |
| *** HenryG has quit IRC | 22:15 | |
| clarkb | jhesketh: we could, it would make the inheritance a bit more ugly though as I need to preset config in the new class | 22:15 |
| clarkb | jhesketh: so it would need to be ifdef'd like make_dirs | 22:15 |
| *** HenryG_ has quit IRC | 22:15 | |
| clarkb | jhesketh: I am beginning to think that I should rip out a third class and haev a common base one that doesn't do much | 22:15 |
| clarkb | jhesketh: let me give that whirl | 22:16 |
| *** HenryG has joined #openstack-infra | 22:16 | |
| clarkb | jhesketh: actually ugh, so much refactoring would need to be done to do that too | 22:16 |
| jhesketh | ah I see | 22:16 |
| jhesketh | yeah, I don't think a base class would be trivial | 22:16 |
| jhesketh | I don't mind how it is, I'm just not a fan of class variables | 22:17 |
| jhesketh | anyway, I'll review it properly soon :-) | 22:17 |
| clarkb | jhesketh: great thank you | 22:19 |
| clarkb | jhesketh: when I have more time I think refactoring it to do what I described and have a proper base with test classes that look like my new test class would be best | 22:19 |
| clarkb | jhesketh: but that is going to be a much larger change | 22:19 |
| *** krtaylor has quit IRC | 22:22 | |
| *** chadlung has quit IRC | 22:23 | |
| *** changbl has quit IRC | 22:23 | |
| *** rwsu has joined #openstack-infra | 22:26 | |
| *** weshay has quit IRC | 22:26 | |
| *** dstanek has quit IRC | 22:26 | |
| *** UtahDave has joined #openstack-infra | 22:27 | |
| *** chuck_ has joined #openstack-infra | 22:27 | |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added parallelization options https://review.openstack.org/75514 | 22:28 |
| mikal | anteaya: what do you think of this? Its a new prototype of the third party CI report: http://dev2.ingenious.com.au/nova-cireport_templated.html | 22:30 |
| *** sandywalsh is now known as sw_away | 22:32 | |
| *** prad has quit IRC | 22:32 | |
| *** thuc has quit IRC | 22:32 | |
| *** thuc has joined #openstack-infra | 22:32 | |
| *** esker has quit IRC | 22:33 | |
| *** esker has joined #openstack-infra | 22:34 | |
| msabramo | https://review.openstack.org/84950 | 22:34 |
| *** dkehn has quit IRC | 22:35 | |
| *** melwitt has quit IRC | 22:35 | |
| *** thomasem has joined #openstack-infra | 22:36 | |
| *** thuc has quit IRC | 22:37 | |
| jhesketh | clarkb: yep, agreed | 22:37 |
| *** esker has quit IRC | 22:38 | |
| jeblair | fungi, pleia2: so the sub requests are a function of demand and capacity. and i think we're just seeing it at the edge of the constraints here | 22:38 |
| jeblair | the formula at work is: | 22:38 |
| jeblair | (demand_for_precise / total_tripleo_demand) * available_tripleo_nodes | 22:39 |
| jeblair | currently available_tripleo_nodes is 0, so actually that's why the subrequest shows up as 0; that's not as alarming as i thought it was earlier | 22:39 |
| pleia2 | ah, I see | 22:40 |
| *** jlk has joined #openstack-infra | 22:40 | |
| *** jlk has left #openstack-infra | 22:40 | |
| msabramo | http://logs.openstack.org/50/84950/2/gate/gate-jenkins-job-builder-pep8/4a9b71e/console.html - Couldn't find index page for 'pbr' (maybe misspelled?) - WTF | 22:40 |
| jeblair | but even if we had one node available, it would be, under current conditions: 160/161*1 | 22:40 |
| jhesketh | fungi: Howdy | 22:40 |
| *** alex-gone is now known as Alexandra | 22:41 | |
| jeblair | pleia2: so basically if you take the output of that, and it ever rounds to a value that is < the available nodes, we'll get an f20 node. | 22:41 |
| fungi | jeblair: got it, so we probably just need to special case values between 0 and 1 node to round up to 1? | 22:41 |
| fungi | from a principle of least surprise standpoint | 22:41 |
| jeblair | fungi: not necessarily; i'm not sure the current behavior is wrong... | 22:41 |
| *** denis_makogon has quit IRC | 22:42 | |
| fungi | fair enough | 22:42 |
| *** zehicle_at_dell has joined #openstack-infra | 22:42 | |
| jeblair | i mean, it feels wrong because we haven't had an f20 node all day. but the cloud has been down all day. :) | 22:42 |
| fungi | jeblair: well, the main issue is coupling it with jenkins gearman behavior | 22:42 |
| *** gokrokve_ has quit IRC | 22:42 | |
| fungi | jeblair: since the masters have never seen a tripleo-f20 node they're not registering the label, and this have no appreciable demand beyond the min-ready count | 22:43 |
| *** e0ne has quit IRC | 22:43 | |
| jeblair | true, it's not actually seeing the demand | 22:43 |
| clarkb | right, I think we may need to special case 1 just to get a slave to register and record demand | 22:43 |
| jeblair | or we could make the job non-voting | 22:44 |
| pleia2 | we agreed last night that we could make the f20 ones non-voting for now | 22:44 |
| jeblair | we could also add an option to zuul to assume all jobs exist | 22:44 |
| *** yamahata has quit IRC | 22:44 | |
| pleia2 | since it's new and scary anyway | 22:44 |
| pleia2 | I can do that patch now if we want | 22:45 |
| *** zehicle has quit IRC | 22:45 | |
| fungi | how about let zuul treat nonexistent jobs automatically as non-voting? | 22:45 |
| jeblair | i added the registration check as protection against typos, but we have a gate job that blocks those so we don't actually need the check | 22:45 |
| fungi | good point | 22:45 |
| *** jaypipes has joined #openstack-infra | 22:45 | |
| jeblair | pleia2: sounds good. increasing the min_ready a bit is probably a good idea as well | 22:46 |
| jeblair | (it'll mean we come out of the corner case faster) | 22:46 |
| pleia2 | jeblair: is this enough, or should we do more? https://review.openstack.org/#/c/85160/ | 22:46 |
| fungi | pleia2: not yet, until the demand for tripleo-precise nodes falls to half what it is now | 22:47 |
| fungi | i suspect you need it up around 4 or 5 to beat the odds | 22:47 |
| fungi | unless it's rounding to the nearest integer instead of rounding down | 22:47 |
| fungi | in which case that might be close | 22:48 |
| jeblair | pleia2: i think that's a good value | 22:48 |
| jeblair | fungi: it rounds to nearest | 22:48 |
| fungi | oh, then yeah, soonish min-ready 2 should bring it around | 22:48 |
| *** mattymo has quit IRC | 22:48 | |
| jeblair | pleia2: a demand ratio of 5:1 means that you will get at least one f20 node as long as there are 3 nodes available | 22:49 |
| *** mattymo has joined #openstack-infra | 22:49 | |
| fungi | also https://review.openstack.org/83057 might fix this faster | 22:49 |
| jeblair | fungi: indeed, is it up? | 22:49 |
| fungi | my tests say yes | 22:49 |
| jeblair | fungi: that lgtm, should i aprv? | 22:50 |
| fungi | though i haven't tried to nova boot in it yet, or checked the quotas. lemme do that first | 22:50 |
| fungi | just a sec | 22:50 |
| fungi | i mainly just looked to see that it was responding and the images and flavors were suitable | 22:50 |
| fungi | i should probably boot an instance and ssh into it | 22:50 |
| jeblair | clarkb: you didn't respond to my comment on https://review.openstack.org/#/c/76057/4..7/tests/test_scheduler.py ? | 22:52 |
| *** marun_afk is now known as marun | 22:52 | |
| clarkb | jeblair: no, beacuse I stopped doing location overrides for all tests | 22:53 |
| openstackgerrit | A change was merged to openstack-infra/config: Increase min-ready for tripleo-f20 nodes https://review.openstack.org/85160 | 22:53 |
| clarkb | jeblair: which made that code valid | 22:53 |
| jeblair | clarkb: i think getting rid of a faked component is a wonderful thing. why wouldn't we want to do it? | 22:53 |
| *** atiwari has quit IRC | 22:54 | |
| clarkb | jeblair: so, the latest patchset runs tests as we always have for zuul, then it runs them again with some different options set. I think that makes it necessary to have that faked component | 22:54 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Make check-tripleo-overcloud-f20 non-voting https://review.openstack.org/85194 | 22:54 |
| *** dstanek has joined #openstack-infra | 22:54 | |
| clarkb | jeblair: the reason for that is to test the poll and remote fetch path as well as the other paths | 22:54 |
| jeblair | clarkb: i don't believe that the fetch/non-fetch paths are any different as far as the tests go | 22:55 |
| jeblair | clarkb: the only thing that changes is this https://review.openstack.org/#/c/76057/7/zuul/trigger/gerrit.py | 22:55 |
| jeblair | clarkb: and that entire function is the thing that's faked out | 22:55 |
| jeblair | so there is no difference for fetch | 22:55 |
| clarkb | jeblair: there is when fetch_url isn't used | 22:56 |
| clarkb | jeblair: I can set fetch_url in all tests if you want | 22:56 |
| * ttx lurks in the night | 22:56 | |
| jeblair | clarkb: no wait let's not jump to doing anything... | 22:56 |
| jeblair | 22:59 < clarkb> jeblair: there is when fetch_url isn't used | 22:56 |
| jeblair | clarkb: can you show me where there's more than one code path through zuul on whether fetch_url is or isn't used? | 22:57 |
| *** jlk has joined #openstack-infra | 22:57 | |
| pleia2 | ttx: I'd appreciate your feedback on the horrible things I'm doing to your bugdaystats ;) https://review.openstack.org/#/q/status:open+project:openstack-infra/bugdaystats,n,z | 22:57 |
| clarkb | jeblair: in you pointed the big one out. The other is in the tests themselves. EG if we don't fake the path you pointed out we won't get the right path without the fake | 22:57 |
| *** zehicle_at_dell has quit IRC | 22:57 | |
| clarkb | s/in// | 22:57 |
| pleia2 | ttx: (it's actually not much, just style cleanup once we get it fully moved to infra, another pending review) | 22:58 |
| ttx | pleia2: could you fire me an email about it ? Otherwise I won't remember tomorrow | 22:58 |
| pleia2 | ttx: will do! | 22:58 |
| jeblair | clarkb: right, but we're not testing the tests. we're only testing zuul. so the only code paths we need to make sure we're covering are in zuul | 22:58 |
| clarkb | jeblair: we can get the correct path in teh tests without the fake if we set fetch_url for all tests | 22:58 |
| fungi | ttx: did you see adam_g's -announce ml moderation request (his new address apparently isn't whitelisted) | 22:58 |
| jeblair | clarkb: and the only code path in zuul affected by that is opaque to the tests because that whole method is faked | 22:58 |
| *** signed8bit has quit IRC | 22:59 | |
| ttx | fungi: looking | 22:59 |
| clarkb | jeblair: let us rewind | 22:59 |
| *** dstanek has quit IRC | 22:59 | |
| jeblair | clarkb: ok | 22:59 |
| ttx | fungi: accepted | 22:59 |
| fungi | ttx: adam_g will be pleased | 23:00 |
| *** saschpe has quit IRC | 23:00 | |
| clarkb | jeblair: patchset 4 assumed one config for testing. latest patchset assumes more than one and only adds new tests, it does not change existing tests. You are making an argument for removing that but I think doing so is orthogonal to this change | 23:00 |
| clarkb | *removing that fake | 23:00 |
| clarkb | if we decide to remove it (because it does seem like a reasonable idea) we will need to add fetch_url to all of the configs used in testing | 23:01 |
| *** david-lyle has quit IRC | 23:01 | |
| *** dcramer_ has quit IRC | 23:01 | |
| *** khyati has quit IRC | 23:02 | |
| *** CaptTofu has joined #openstack-infra | 23:02 | |
| *** dizquierdo has quit IRC | 23:02 | |
| clarkb | jeblair: if you would like a follow up change that removes the fake I can do that, or if you prefer I can squash it together | 23:03 |
| jeblair | clarkb: yes. in patchset 4 my suggestion was an optimization because you were setting it everywhere; in patchset 7 the question changes slightly because it's part of the second pass of tests; i think removing it eliminates the need for half of that. i haven't thought about poll yet. | 23:04 |
| *** yamahata has joined #openstack-infra | 23:04 | |
| jeblair | clarkb: so let's move to that, if you don't mind; why do you think poll needs all of the tests run again? | 23:04 |
| clarkb | jeblair: because I couldn't come up with a good way of running a specific test under different configuration | 23:05 |
| clarkb | jeblair: since so much of the config stuff is prebaked | 23:05 |
| adam_g | ttx, thanks! | 23:05 |
| *** matsuhashi has joined #openstack-infra | 23:05 | |
| jeblair | clarkb: oh, because you need to launch a merger with the poll option set, and that happens in setUp? | 23:06 |
| clarkb | jeblair: yup | 23:06 |
| clarkb | jeblair: all solutions I came up with required a new class and the existing one does 99% of the setup that si required already | 23:06 |
| clarkb | so I am piggy backing off of that | 23:06 |
| *** khyati has joined #openstack-infra | 23:07 | |
| clarkb | jeblair: this comes back to the end of my discussion with jhesketh. The best way to do this would be to completely refactor the "base" test class. It can do all of the fake setups and config things. Then other classes can mix that in and define particular tests | 23:07 |
| jeblair | clarkb: ok, so i like the idea of refactoring that setup method into something more reusable, so you could have a class for one of several different scenarios | 23:08 |
| jeblair | clarkb: ++ that ^ | 23:08 |
| jeblair | clarkb: i wonder though if in the mean time, we could avoid running all the tests twice | 23:08 |
| *** zehicle_at_dell has joined #openstack-infra | 23:08 | |
| jeblair | clarkb: one possibility might be to stop and start the merger in the poll test | 23:09 |
| *** dkehn has joined #openstack-infra | 23:09 | |
| clarkb | jeblair: the other issue is CONFIG is global in TestScheduler so all tests run after a config update will get that modified config (this can be worked around more simply I think) | 23:10 |
| clarkb | jeblair: why don't I give the end state a shot since I think it will be simple to do | 23:11 |
| clarkb | jeblair: then we don't have to worry about the intermediate state | 23:11 |
| clarkb | jeblair: simple to do, but harder to review | 23:11 |
| jeblair | clarkb: ok; i like that idea too, but was hesitant to suggest it | 23:11 |
| clarkb | because I expect the diff to be ugly | 23:11 |
| jeblair | clarkb: will you do the refactor as one patch, then rebase 76057 on it? | 23:12 |
| *** thedodd has quit IRC | 23:12 | |
| clarkb | rebase 76057 on it | 23:12 |
| Steap | Hey! python-heatclient just got Python 3 compatibility; could someone merge https://review.openstack.org/#/c/85170/ so that the py3 becomes voting before Python3-incompatible patches are merged? | 23:12 |
| *** UtahDave has quit IRC | 23:12 | |
| *** whoops has quit IRC | 23:12 | |
| jeblair | clarkb: okay, thanks; i'll finish reviewing 76057 now | 23:13 |
| jhesketh | jeblair: added a couple of inline comments to 84873,2 | 23:14 |
| clarkb | Steap: done | 23:14 |
| jeblair | jhesketh: thx | 23:14 |
| *** nkinder has joined #openstack-infra | 23:15 | |
| *** e0ne_ has joined #openstack-infra | 23:15 | |
| openstackgerrit | A change was merged to openstack-infra/config: heatclient: make the py33 gate voting https://review.openstack.org/85170 | 23:18 |
| jeblair | clarkb: lgtm otherwise, i only had comments we discussed here. | 23:18 |
| *** jgrimm has quit IRC | 23:18 | |
| fungi | jeblair: i've successfully booted an instance, created and added a floating ip, confirmed i could reach the ssh port on it and got a host key back, and deleted the instance and floating ip all without a hitch... i wish the tripleo hp region were up though so i could compare quotas, but regardless they're way too low. i'll -1 and leave comments | 23:19 |
| jeblair | fungi: :( thanks | 23:19 |
| *** saschpe has joined #openstack-infra | 23:19 | |
| fungi | otherwise, aside from capacity, it looks good! :/ | 23:19 |
| *** e0ne_ has quit IRC | 23:20 | |
| clarkb | jeblair: thanks | 23:20 |
| *** dstanek has joined #openstack-infra | 23:21 | |
| Steap | clarkb: thx | 23:23 |
| fungi | got my hopes up... *sigh* | 23:27 |
| fungi | the summary is that the instances, floating ips, vcpus and ram all are way below the 20 max-servers in that change | 23:28 |
| fungi | the worst is vcpus... we have enough to be able to boot two of the flavor specified in the nodepool.yaml, and those will be needed just to build their two images each day | 23:29 |
| *** markwash has quit IRC | 23:29 | |
| pleia2 | fungi: oops, thanks for looking into that | 23:29 |
| *** german_ has quit IRC | 23:30 | |
| fungi | pleia2: no problem--it's something we should always check when making capacity changes like that, or bringing new providers on line. sorry i didn't think to do so sooner | 23:30 |
| *** HenryG_ has joined #openstack-infra | 23:30 | |
| *** german_ has joined #openstack-infra | 23:30 | |
| * pleia2 nods | 23:30 | |
| pleia2 | all I did was check basic stuff with my user account to make sure it was responsive | 23:30 |
| *** markmcclain has quit IRC | 23:31 | |
| fungi | pleia2: future tip... nova absolute-limits | 23:31 |
| pleia2 | thanks | 23:31 |
| fungi | (plus a little math matching up to the nova flavor-list entry you're planning to use) | 23:31 |
| *** zehicle_at_dell has quit IRC | 23:32 | |
| *** HenryG has quit IRC | 23:32 | |
| *** zehicle_at_dell has joined #openstack-infra | 23:32 | |
| fungi | divide cores into max cores, ram into max ram, make sure you have high enough max floating-ips for the max instances, et cetera | 23:32 |
| * pleia2 nods | 23:32 | |
| fungi | whichever is lowest, that's your effective max node count (barring hidden limits like the number of precreated tripleo work environments) | 23:33 |
| *** Sukhdev has joined #openstack-infra | 23:34 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gear: Add access control https://review.openstack.org/84873 | 23:35 |
| jeblair | jhesketh: okay, that addresses exceptions | 23:35 |
| jeblair | jhesketh: i have not addressed or replied to 'subject regex' and 'disconnect certificates with a cn of self' | 23:36 |
| jeblair | jhesketh: i think irc may be better for that | 23:36 |
| clarkb | jeblair: running into trouble with test_statsd() is that meant to use the fake statsd eg self.statsd or should it actually do the extras import there? | 23:36 |
| jeblair | jhesketh: i think the subject thing is really a use-case vs efficiency thing | 23:36 |
| jhesketh | jeblair: sure, I'm not set on either of those features | 23:37 |
| asselin | Hi, I found the source of the dns issues I'm running into: in the puppet "unbound" class. What's the simplest way to avoid the changes made there? | 23:37 |
| clarkb | jeblair: nevermind, pep8 was being obtuse | 23:37 |
| *** thuc has joined #openstack-infra | 23:37 | |
| *** HenryG has joined #openstack-infra | 23:38 | |
| jhesketh | jeblair: I hadn't considered efficiency, that might be a good point... although if we keep the regex compiled in the ACL entry the overhead might not be too bad | 23:38 |
| jeblair | clarkb: i _think_ that's meant to actually test statsd itself, since we usally fake it | 23:38 |
| *** thuc has quit IRC | 23:38 | |
| *** zns has quit IRC | 23:38 | |
| *** yamahata has quit IRC | 23:38 | |
| jhesketh | asselin: Why avoid the changes and not fix them? | 23:39 |
| jeblair | clarkb: except we don't fake it do we? | 23:39 |
| jeblair | clarkb: honestly, that's starting to seem a little weird | 23:39 |
| *** thuc has joined #openstack-infra | 23:39 | |
| *** thuc has quit IRC | 23:39 | |
| jeblair | clarkb: (we have a fake server, but we use the real api; zuul itself uses those calls... maybe there was a point where we weren't exercising all of them? maybe we still aren't?) | 23:39 |
| clarkb | jeblair: ya, some stuff uses fakes other stuff uses the fake server | 23:39 |
| clarkb | jeblair: I think they are both used in different ways :) | 23:39 |
| clarkb | asselin: does 8.8.8.8 not resolve dns for you? | 23:40 |
| asselin | jhesketh, good question. I don't know how to 'fix'. The comments int he file suggest rackspace needs those. | 23:40 |
| *** thuc has joined #openstack-infra | 23:40 | |
| clarkb | asselin: the idea with unbound is that we get better DNS reliability if we add a DNS cache to each slave | 23:40 |
| asselin | # Rackspace uses static config files file { '/etc/resolv.conf': content => "nameserver 127.0.0.1\n", | 23:40 |
| *** HenryG_ has quit IRC | 23:41 | |
| jeblair | jhesketh: yeah, my thought was to try to keep acl lookups as fast as possible; i know we'll need regex for names, but i'm not expecting to apply matching to subject names, so i tried to keep it simple | 23:41 |
| jeblair | jhesketh: i agree it could be useful; maybe we can defer until we have a solid use case? | 23:41 |
| asselin | clarkb, jhesketh the rackspace settings applied break dns on my slave node | 23:42 |
| jeblair | asselin: the module attempts to configure the host to use the nameserver it's installing regardless of the varying configurations we have found on provider images | 23:42 |
| jeblair | asselin: what's broken about it? | 23:42 |
| *** HenryG has quit IRC | 23:42 | |
| *** zehicle_at_dell has quit IRC | 23:42 | |
| *** zehicle_at_dell has joined #openstack-infra | 23:43 | |
| asselin | jeblair, as soon as those settings are applied to /etc/resolv.conf, dns stops working | 23:43 |
| jeblair | asselin: what os are you using? | 23:43 |
| asselin | jeblair, ubuntu | 23:43 |
| jeblair | asselin: precise? | 23:43 |
| asselin | jeblair, yes 12.04 dns works perfectly up until this part of the script | 23:44 |
| jhesketh | jeblair: sure, happy for it to come in later if we find we want it :-) | 23:44 |
| jeblair | asselin: after it fails, is the 'unbound' process running? | 23:45 |
| *** unicell has joined #openstack-infra | 23:45 | |
| *** dcramer_ has joined #openstack-infra | 23:45 | |
| *** esker has joined #openstack-infra | 23:45 | |
| jeblair | asselin: and if so, what is the output of "dig +dnssec -t soa com. @127.0.0.1" | 23:46 |
| *** homeless has quit IRC | 23:46 | |
| asselin | jeblair, I will try now and get back to you. | 23:46 |
| jeblair | jhesketh: so the other thing; connections from a cert named 'self'... | 23:47 |
| jhesketh | jeblair: yes, so my thought there is more from a usability point of view than security | 23:47 |
| jeblair | jhesketh: i agree it seems like we could do something there. should we reject those on connect, or just error out if you try to run acl commands in such a context? | 23:48 |
| *** mriedem has quit IRC | 23:48 | |
| jhesketh | jeblair: reject on connect | 23:48 |
| jeblair | jhesketh: if we do that, we can't provide feedback to the user, but we can log it for the admin | 23:48 |
| jhesketh | because the problem comes when somebody else tries to grant permissions for the person with the cert named "self" | 23:48 |
| jeblair | presumably the admin is signing the certs anyway, so that's probably ok | 23:48 |
| jhesketh | they think they're granting for "self" but they are actually only modifying their own | 23:48 |
| *** CaptTofu has quit IRC | 23:49 | |
| jhesketh | jeblair: well we could allow the connection, send back a message, then disconnect | 23:49 |
| jhesketh | but that's probably not within the protocol spec | 23:49 |
| jhesketh | (actually is ACL in the spec?) | 23:49 |
| jeblair | jhesketh: not so much | 23:49 |
| jeblair | jhesketh: nope. i'm making it up as i go along. but i did talk to brian aker about it and he's on board with the general idea. | 23:49 |
| jhesketh | okay | 23:50 |
| jeblair | jhesketh: the admin protocol is much looser than the binary protocol anyway | 23:50 |
| jhesketh | I'm fine with just disconnecting them and documenting it so that when somebody is confused hopefully they consult the manual | 23:50 |
| *** wenlock has quit IRC | 23:51 | |
| *** mrodden has quit IRC | 23:51 | |
| jhesketh | jeblair: either that or we change the command | 23:51 |
| jhesketh | eg: "grant-self" | 23:51 |
| fungi | jeblair: so are acls part of the gearman protocol, or a gear-only extension? | 23:56 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Tasks are now editable https://review.openstack.org/84888 | 23:57 |
| *** mdenny has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
