| mordred | nibalizer: which is that ansible users who happen to upgrade python-neutronclient are completely broken right now | 00:00 |
|---|---|---|
| mordred | nibalizer: maybe I should have led with that | 00:00 |
| *** rbrndt has quit IRC | 00:00 | |
| clarkb | Error: Could not stat device /dev/vdc - No such file or directory. the mystery deepens | 00:00 |
| mordred | nibalizer: we had a bug in deserialization that got tickled by them changing something slightly in their output format | 00:00 |
| nibalizer | because of that 'to-munch-or-not-to-munch patch?' | 00:00 |
| mordred | nibalizer: not really- although it is munch related | 00:01 |
| *** kzaitsev_mb has joined #openstack-infra | 00:01 | |
| mordred | nibalizer: it's that we had things doing if type(obj) == dict ... rather than isinstance(obj, dict) | 00:01 |
| mordred | nibalizer: which means when they started returning a dict subclass instead of a dict | 00:01 |
| nibalizer | yea thats what i was talking about | 00:01 |
| mordred | we went boom | 00:01 |
| nibalizer | perhaps to obiqueley | 00:01 |
| mordred | yeah. then YES | 00:01 |
| mordred | there is another patch that discusses whether or not we shoudl really be munching | 00:01 |
| jeblair | i like how you are almost communicating with each other :) | 00:01 |
| *** dbrycen has quit IRC | 00:02 | |
| jeblair | it's friday beer time clearly | 00:02 |
| *** ddieterly has quit IRC | 00:02 | |
| bkero | By beer you mean deploy to production | 00:02 |
| clarkb | hrm --keep isn't useful because I don't have an ssh key to log in with yet | 00:03 |
| clarkb | fun | 00:03 |
| clarkb | silly clouds | 00:03 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add zlib1g-dev to bindep-fallback.txt https://review.openstack.org/288805 | 00:03 |
| * clarkb finds the private key | 00:03 | |
| nibalizer | mordred: when did they python-neutronclinet release happen? | 00:03 |
| fungi | clarkb: pabelanger: ^ so far that's the only one i find missing going back through modules/openstack_project/manifests/jenkins_params.pp additions as far as 1.5 years (just to be safe) | 00:03 |
| fungi | so it was really just the one change which slipped through adding libjpeg-dev and zlib1g-dev | 00:04 |
| pabelanger | fungi: WFM | 00:04 |
| pabelanger | fungi: ya, that is pretty good | 00:04 |
| mordred | nibalizer: this week | 00:05 |
| mordred | nibalizer: like 2 days ago? | 00:05 |
| fungi | also the state of the bindep fallback list has gotten pretty terrible. i'm going to do a cleanup pass on it (references to platforms we don't run or being unnecessarily specific on platform matches, or making redundant entries... it's well longer than it needs to be) | 00:05 |
| *** e0ne has quit IRC | 00:05 | |
| nibalizer | mordred: how long has it been since a shade release? | 00:05 |
| mordred | heh | 00:06 |
| mordred | yup | 00:06 |
| mordred | nibalizer: http://paste.openstack.org/show/489413 | 00:06 |
| mordred | nibalizer: that long | 00:06 |
| *** ashleighfarnham has quit IRC | 00:06 | |
| mordred | Shrews: we should maybe release more frequently | 00:06 |
| nibalizer | mordred: yea that does not look like a friday release | 00:07 |
| *** rguillebert has quit IRC | 00:07 | |
| *** annegentle has quit IRC | 00:07 | |
| *** ashleighfarnham has joined #openstack-infra | 00:07 | |
| nibalizer | if you want to make a branch and cherry pick the bare minimum and cut a 1.4.1 im cool with it though | 00:07 |
| clarkb | actually looks like maybe my lvm2 intssall failed I think their imgae must be specialer | 00:08 |
| * clarkb uploads an iage | 00:08 | |
| nibalizer | looking at the diff | wc -l is 4k lines | 00:08 |
| jeblair | clarkb: are we also waiting on the swift changes pending a shade release? | 00:09 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Sort bindep-fallback.txt https://review.openstack.org/288810 | 00:09 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Clean up bindep-fallback.txt https://review.openstack.org/288811 | 00:09 |
| fungi | pabelanger: ^ | 00:09 |
| mordred | jeblair: ya. the swift changes are also pending a shade release | 00:10 |
| *** Jeffrey4l has quit IRC | 00:10 | |
| fungi | so not as terrible as i thought | 00:10 |
| Shrews | mordred: i've been nagging about releasing. perhaps you should teach me release stuff at some point? | 00:11 |
| nibalizer | Shrews: its git tag 'tag'; git push --tags | 00:12 |
| *** thorst has joined #openstack-infra | 00:12 | |
| pabelanger | fungi: I am not sure liberasurecode-devel is a thing on centos | 00:12 |
| nibalizer | there is video of mordred doing it at his real slim shade talk in tokyo | 00:12 |
| pabelanger | fungi: but gate will tell us | 00:12 |
| jeblair | nibalizer, Shrews: "git tag -s <tag>" | 00:12 |
| fungi | pabelanger: well, it was being applied to centos-7 previously | 00:12 |
| nibalizer | ah yes, that | 00:12 |
| clarkb | I am uploading the same image that I used in vexxhost to osic | 00:12 |
| jeblair | don't forget the -s | 00:12 |
| nibalizer | ya it needs the s | 00:12 |
| clarkb | to avoid debconf being unhappy when trying to apt-get install -y | 00:12 |
| nibalizer | not your -s, but just any old -s | 00:12 |
| openstackgerrit | Merged openstack-infra/system-config: Use osic-cloud1 https://review.openstack.org/288793 | 00:12 |
| fungi | pabelanger: that change just condenses fedora22+fedora23+centos7 to rpm | 00:13 |
| Shrews | nibalizer: jeblair: by 'learn' i mean, give me permission and let me know if there are release rules | 00:13 |
| jeblair | nibalizer: though i will lend mine if needed | 00:13 |
| pabelanger | fungi: Oh, ya. | 00:13 |
| nibalizer | mtreinish: did we get the subunit worker thing? | 00:13 |
| pabelanger | fungi: guess it is time to call it a day | 00:13 |
| * nibalizer goes to look | 00:13 | |
| Shrews | i know tagging does the magic | 00:13 |
| fungi | nibalizer: also git push --tags is dangerous | 00:14 |
| fungi | nibalizer: push a specific tag by name | 00:14 |
| fungi | nibalizer: we have the workflow documented! http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release | 00:14 |
| nibalizer | is that with git push <tagname> ? | 00:14 |
| pabelanger | have a good weekend all | 00:14 |
| fungi | good weekend pabelanger! | 00:15 |
| jeblair | Shrews, mordred: ah that's fungi's department | 00:15 |
| jeblair | pabelanger: o/ | 00:15 |
| clarkb | if I get this mirror built soonish are we going to make nodepool go? we totally should :) | 00:15 |
| jeblair | clarkb: ++ | 00:15 |
| jeblair | cody-somerville: i'm trying out your change :). apparently the font i'm using does not have the unicode clock, though it does have the triangle. | 00:16 |
| cody-somerville | jeblair: doh. What font are you using? | 00:17 |
| fungi | Shrews: i'm not opposed to you having shade-release membership if you've been wanting to release it and getting up with someone to do it has been getting in the way of progress. it's mostly that we keep the -release groups constrained because if you don't exercise the skill often, you have a higher chance of creating a broken release (which is sometimes worse than not releasing often) | 00:17 |
| clarkb | apparently when we fail to launch node the cleanup doesn't cleanup floating ips | 00:17 |
| * clarkb manually cleans those out | 00:17 | |
| mordred | clarkb: oh nice | 00:18 |
| mordred | clarkb: we have a flag for that in shade | 00:18 |
| clarkb | ya Ithink it just needs updating | 00:18 |
| clarkb | since it is the shade launch node | 00:18 |
| mordred | clarkb: you can just say "cloud.delete_server(server_name, delete_ips=True) | 00:18 |
| fungi | jeblair: most terminals will composit glyphs for missing codepoints from other fontsets you have installed | 00:18 |
| mordred | hrm | 00:19 |
| mordred | clarkb: the script says delete_ips=True | 00:19 |
| mordred | this does not happy me | 00:19 |
| fungi | jeblair: so your preferred font doesn't need to have it as long as you have a font which has it (though that behavior may depend on your font libraries and config) | 00:19 |
| mordred | Shrews: not sure if you know about launch/shade-launch-node.py in system-config | 00:19 |
| clarkb | ubuntu provided image totally works so the one provided by osic has a different debconf for some reason | 00:19 |
| mordred | Shrews: but clark seems to have found a place where it's not doing what we expect | 00:20 |
| Shrews | fungi: i only offer to take load off of mordred. i don't care either way | 00:20 |
| clarkb | oh I take that back what is going on | 00:20 |
| Shrews | mordred: sad making | 00:20 |
| Shrews | should i pour another scotch before here said sad causing thing? | 00:20 |
| Shrews | s/here/hearing/ | 00:20 |
| mordred | Shrews: you should always pour scotch | 00:21 |
| mordred | Shrews: or japanese whiskey | 00:21 |
| Shrews | mmm, nikka | 00:21 |
| mordred | heck yes | 00:21 |
| Clint | correct japanese whiskey | 00:21 |
| jeblair | cody-somerville, fungi: i'm using 6x13. | 00:21 |
| fungi | Shrews: mordred: i've personally avoided trying to release shade myself because i've not had time to figure out if i'm doing the things reno wants of me, or simply preparing to anger it | 00:21 |
| clarkb | mordred: its probably because the create works but then later we explode and don't roll back | 00:21 |
| clarkb | mordred: because we do things like attempt to mount the cinder volume and fail | 00:22 |
| clarkb | bceause there is no /dev/vdc despite asking nova/cinder to attach at that device | 00:22 |
| jeblair | "urxvt -fn 6x13" | 00:22 |
| mordred | fungi: we do not use reno in shade the way other things do | 00:22 |
| mordred | fungi: so you're safe | 00:22 |
| fungi | mordred: er, even more daunting? | 00:22 |
| mordred | clarkb: but - the except does a delete_server | 00:22 |
| fungi | how do you use reno then? | 00:23 |
| mordred | fungi: we put it in the normal sphinx docs | 00:23 |
| mordred | fungi: becuase we do not publish to releasenotes.o.o | 00:23 |
| mordred | since that would be weird | 00:23 |
| clarkb | mordred: oh hrm I dunno then | 00:23 |
| mordred | fungi: http://docs.openstack.org/infra/shade/releasenotes.html | 00:23 |
| jeblair | mordred: what's http://releasenotes.openstack.org/ ? | 00:23 |
| jeblair | i'm having an argument with dns about its existence | 00:24 |
| mordred | or, whereever they go | 00:24 |
| fungi | oh. is this one of the libraries that's annoying package maintainers because of reno scraping git history when they try to generate project documentation? | 00:24 |
| jeblair | i'm losing | 00:24 |
| fungi | jeblair: might be releases.openstack.org | 00:24 |
| Shrews | jeblair: try saying bad things about dns's hands | 00:24 |
| clarkb | I am trying again without specifying the device | 00:24 |
| Shrews | jeblair: b/c that seems to work in GOP debates | 00:24 |
| jeblair | Shrews: i think i will lose harder | 00:25 |
| fungi | jeblair: actually i don't know if bitmap fonts will have a similar behavior. i'm using truetype rendering since i wanted fonts with good east asian codepoint coverage | 00:25 |
| clarkb | nope config drive really does appear to be at vdb | 00:25 |
| jeblair | mordred, fungi: it looks like releases.o.o is what mordred meant, but the actual release notes end up at eg: http://docs.openstack.org/releasenotes/cinder/liberty.html | 00:25 |
| clarkb | I wonder if cinder isn't working here | 00:25 |
| clarkb | or cinder + nova | 00:25 |
| clarkb | I am going to try manually | 00:26 |
| jeblair | fungi: yeah, i use bitmap fonts in terminals; i have a 600dpi threshold for tolerance of antialiasing (which is, incidentally, and totally coincidentally, pretty close to when you don't need it anymore) | 00:27 |
| fungi | heh | 00:27 |
| Shrews | clarkb: mordred: i noticed the recent release of cinder is more strict when deleting things. i.e., we first have to delete snapshots before deleting the volume (unless using a new --cascade option). wondering if the ip thing is similar | 00:27 |
| Shrews | (if i grok the scrollback correctly) | 00:28 |
| mtreinish | nibalizer: I don't think so, the puppet patch still needs a +A | 00:28 |
| mtreinish | after that I think it should be good to go | 00:28 |
| mordred | Shrews: ah. so we might be failing in the delete | 00:29 |
| cody-somerville | jeblair: Hmm... you might be able to upgrade your font. It looks like 6x13 (misc-fixed) supports CEN MES-3A European Unicode 3.2 Subset which should have it. | 00:29 |
| nibalizer | any other cores want to look at https://review.openstack.org/#/c/288706/ | 00:30 |
| fungi | at some point in the past i discovered that certain sans-serif truetype fonts are easier on my dyslexia and actually work well monospaced, or else i too would probably still be using traditional x11 bitmap fonts | 00:30 |
| clarkb | mordred: Shrews fwiw the last couple of deletes seemed to work so maybe I was derping | 00:31 |
| cody-somerville | jeblair: The symbol I used is from Unicode 1.1.0. | 00:31 |
| clarkb | I only kept one server but had 3 leaked fips so don't know how that happened | 00:31 |
| *** akshai has joined #openstack-infra | 00:31 | |
| mordred | clarkb, Shrews: so - honestly, the next amount of work I want to do there is to get us from shade-launch-node to ansible-launch-node | 00:31 |
| *** kingia has joined #openstack-infra | 00:32 | |
| fungi | clarkb: are you sure those "leaked: fips weren't already in there from something else? | 00:32 |
| clarkb | fungi: pretty sure, I hadn't used this account (had used jenkins instead of ci) | 00:33 |
| *** akshai has quit IRC | 00:33 | |
| clarkb | ok osic puts config drive on /dev/sr0 | 00:33 |
| clarkb | so not the vd* conflict | 00:33 |
| fungi | i mean, it's possible osic launched something to test the account, then deleted but forgot to clean up the fip | 00:33 |
| clarkb | something else funny is happening | 00:33 |
| clarkb | fungi: oh maybe | 00:33 |
| *** kingia_ has quit IRC | 00:35 | |
| *** dimsum_ has quit IRC | 00:35 | |
| clarkb | writing this here so I don't forget, it would be nice if clouds set their default compute az to the same location as their default else az so that I don't have to constantly specify these things after the fact and rearrange | 00:37 |
| clarkb | ok debugging has happened | 00:41 |
| clarkb | /dev/vdb works except we make_swap on that :/ | 00:41 |
| clarkb | so I am just going to force attach to /dev/vdc to bypass that >_> | 00:41 |
| mordred | clarkb: ++ | 00:42 |
| mordred | (to the az thing) | 00:42 |
| cody-somerville | jeblair: Just downloaded the latest from http://www.cl.cam.ac.uk/~mgk25/ucs-fonts.html and our symbol doesn't show up in the missing characters file for 6x13. | 00:42 |
| *** dingyichen has quit IRC | 00:42 | |
| clarkb | and that still doesn't work | 00:43 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 00:43 | |
| *** Swami has quit IRC | 00:43 | |
| clarkb | its almost like specifying the device path isn't a thing | 00:43 |
| clarkb | I am going to ninja around this by just not make swapping | 00:43 |
| *** Swami_ has quit IRC | 00:43 | |
| mordred | cody-somerville: there was a symbol? | 00:43 |
| mordred | clarkb: I'm about to EOD - however, the thing you're doing right now is a thing I would like for shade/ansible to be able to help with | 00:44 |
| mordred | clarkb: so when you're done, would you send me a short whatever about what went on? | 00:44 |
| clarkb | yup, though relaly there are a couple cloud side issues here | 00:44 |
| mordred | totes | 00:44 |
| clarkb | 1) config drive and cinder volumes can fight over a device and 2) specifying device doesn't work? | 00:45 |
| mordred | I just want to make sure that shade fixes as manyof them as it can | 00:45 |
| mordred | 1) WHAT? | 00:45 |
| mordred | also | 00:45 |
| mordred | 1) WHAT? | 00:45 |
| *** markvoelker has quit IRC | 00:45 | |
| clarkb | mordred: yes basically config drive can go on /dev/vdb and cinder/nova think the volume goes there too | 00:45 |
| clarkb | mordred: the best part is when you reboot the config driv emoves to /dev/vdc and volume goes on /dev/vdb | 00:46 |
| mordred | clarkb: oh. SO... | 00:46 |
| mordred | clarkb: there is a thing that might be useful | 00:46 |
| mordred | clarkb: which is that everywhere except xen (and I think BobBall_AWOL fixed xen) | 00:46 |
| mordred | clarkb: the name you give a cinder volume will show up in the fancy device name volume stuff | 00:47 |
| clarkb | ok got past that particular issue, we might have a mirror in a few minutes | 00:47 |
| clarkb | mordred: under /dev/disk/by-lable? | 00:47 |
| cody-somerville | mordred: Used in my patch to Gertty. I added navigation breadcrumb, see https://review.openstack.org/#/c/288250/1 and for example see http://ibin.co/2Z1FdV0JZvzP | 00:47 |
| mordred | clarkb: yes | 00:47 |
| mordred | clarkb: so if you do "cinder create foo" | 00:47 |
| mordred | clarkb: you sohuld e able to get /dev/disk/by-label/foo | 00:47 |
| clarkb | mordred: that could be helpful except that a cloud shouldn't do the thing it is doing :/ | 00:47 |
| clarkb | which is get confused | 00:47 |
| mordred | clarkb: dude. if a cloud ever did something right ... | 00:48 |
| clarkb | ok this seems to have worked by not running make swap | 00:48 |
| clarkb | and letting cinder/nova assignt the device | 00:48 |
| clarkb | so I have that bit of feedback for osic "cannot specify dvice with cinder attach" | 00:48 |
| mordred | ++ | 00:49 |
| clarkb | btw with all these flavors that don't have an ephemeral disk /dev/vdb is actually something we probably don't want swap on | 00:50 |
| *** Daisy has joined #openstack-infra | 00:50 | |
| clarkb | we might fix that by specifying an explicit swap device? | 00:50 |
| clarkb | behold http://172.99.106.64/ | 00:51 |
| clarkb | I am adding dns now | 00:51 |
| mordred | clarkb: woot! | 00:51 |
| clarkb | actually let me check disk setup first | 00:51 |
| clarkb | looks good adding dns now | 00:52 |
| *** ccrouch1 has quit IRC | 00:52 | |
| clarkb | http://mirror.regionone.osic-cloud1.openstack.org should be working shortly | 00:53 |
| clarkb | once it does any opposition to me approving the thing to add to nodepool? | 00:53 |
| *** ccrouch has joined #openstack-infra | 00:53 | |
| mordred | clarkb: nope | 00:53 |
| jeblair | ++ | 00:54 |
| clarkb | 8.8.8.8 resolves it, I am going to approve nodepool add now | 00:54 |
| clarkb | wow jenkins02 is still going | 00:54 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Follow name_or_id pattern on domain operations https://review.openstack.org/288715 | 00:55 |
| *** roxanaghe has quit IRC | 00:55 | |
| rcarrillocruz | Shrews: ^ | 00:56 |
| Shrews | rcarrillocruz: may i offer an alternative? | 00:58 |
| *** Daisy_ has joined #openstack-infra | 00:58 | |
| mordred | heh. I just accidentally approved a patch in gertty because I thought I was switching to IRC window #3 (A-3) but instead I was in gerrty, so (A-2) approve | 00:59 |
| *** Daisy_ has quit IRC | 00:59 | |
| mordred | luckily, it was a patch that I'd already reviewed and jeblair had already approved :) | 00:59 |
| *** Daisy_ has joined #openstack-infra | 00:59 | |
| Shrews | rcarrillocruz: instead of a new param, why not just always do a get_domain() with whatever domain_id is. then document that domain_id can be either name or id | 00:59 |
| bswartz | hey guys, I have a question about he "scp" publisher in jenkins -- it seems that it directly overwrites existing files, and for the time during the upload, anyone who downloads the file gets something corrupted -- would it be possible to upload to a temp file and then atomically rename the temp file to the permanent name? does jenkins allow that? | 01:00 |
| bkero | mordred: Should invent a way to sign your approvals with your gpg key, that way you need to type your passphrase to do it | 01:00 |
| bkero | no accidental approvals :) | 01:00 |
| jeblair | well, also, you have like half an hour to change your mind and revoke the approval :) | 01:00 |
| clarkb | bswartz: not with the scp plugin | 01:00 |
| clarkb | bswartz: there are alternatives like using rsync or uploading to swift | 01:00 |
| rcarrillocruz | Shrews: i think that'd be more confusing to shade users. Having domain_id a meaning of *domain_id* in all places but on these operations name or id is confusing | 01:01 |
| jeblair | bkero: then i could have gertty cache the passphrase for convenience! | 01:01 |
| *** Daisy has quit IRC | 01:01 | |
| bkero | jeblair: that sounds really convenient. I'll draw up a blueprint asap | 01:02 |
| rcarrillocruz | in other news spacex failed again | 01:02 |
| rcarrillocruz | :/ | 01:02 |
| Shrews | rcarrillocruz: i can see that. but it would also be more consistent with the other apis that accept name or id as the 1st arg | 01:02 |
| Shrews | rcarrillocruz: just tossing out an idea for consideration | 01:03 |
| fungi | bswartz: this is also a known issue with, e.g., branch tarball publication | 01:03 |
| bswartz | clarkb: can I use scp to upload the file and then something else afterwards to do the rename? "ssh mv <temp> <perm>" would work if jenkins can do that | 01:03 |
| fungi | there are times when if you try to retrieve it while it's in the process of being replaced, you get a truncated copy | 01:03 |
| clarkb | bswartz: I think if you go that route just bypass jenkins scp plugin entirely | 01:04 |
| clarkb | and use eg rsync | 01:04 |
| jeblair | rcarrillocruz: the barge is called "Of Course I Still Love You"? they are iain m banks fans! | 01:04 |
| openstackgerrit | Merged openstack-infra/project-config: Use OSIC with nodepool https://review.openstack.org/288584 | 01:04 |
| fungi | bswartz: the scp publisher plug-in in jenkins doesn't really do anything beyond invoke scp | 01:04 |
| rcarrillocruz | Shrews: a 3rd one would be 'create_domain_now_with_name_or_id_yay(self, name_or_id, blah) | 01:04 |
| rcarrillocruz | just kidding :D | 01:04 |
| bswartz | okay well the window of danger is fairly narrow | 01:04 |
| Shrews | rcarrillocruz: YES! so do that | 01:04 |
| * Shrews reaches internet tolerance level. logs off. night all | 01:05 | |
| fungi | clarkb: bswartz: part of the challenge here is that the publishers run in the context of the jenkins master, which is why they're generally feature-poor. they don't run on the worker with access to the original workspace | 01:05 |
| bswartz | we've observed uploads taking from 40-60 seconds, and if we do that on average once per week, the chance of a job getting bitten by that is vanishingly small | 01:05 |
| *** gyee has quit IRC | 01:05 | |
| NobodyCam | fungi: just checking ... good for that recheck? | 01:05 |
| * rcarrillocruz goes to bed with the sweet flavor of green tests on his change, night all | 01:05 | |
| bswartz | also if it gets bad enough we can move our gate tests to rely on tagged qcow files | 01:06 |
| *** sdake has joined #openstack-infra | 01:06 | |
| bswartz | I just wondered if there was an easy way to fix this | 01:06 |
| fungi | NobodyCam: i have no idea what the status of the images is. clarkb: were you the one who kicked off the rebuilds? | 01:06 |
| clarkb | fungi: I did, everywhere but ovh-bhs one should be good | 01:06 |
| clarkb | except still no zlib | 01:07 |
| *** Daisy_ has quit IRC | 01:07 | |
| *** Daisy has joined #openstack-infra | 01:08 | |
| fungi | bswartz: we've got a couple of ideas for improved publication mechanisms. our swift publisher can upload directly from the worker into a permission-constrained location in a swift container, but that needs special frontend care to index it. we also have jobs running on privileged workers that can stick files into afs and serve them from our mirror network, and a design not yet implemented for an | 01:08 |
| fungi | atomic publication pipeline for documentation relying on an afs volume | 01:08 |
| mordred | AFS | 01:08 |
| *** mtanino has quit IRC | 01:09 | |
| * mordred points to the epic win of spinning up new mirror hosts we're experiencing | 01:09 | |
| bswartz | fungi: as long as the image is ultimately downloadable with wget I don't think we care what goes on behind the scenes | 01:09 |
| fungi | clarkb: NobodyCam: 288805 adds zlib1g-dev and its rpm equivalent | 01:09 |
| bswartz | what you describe sounds like overkill for our simple use case though | 01:10 |
| *** Qiming has joined #openstack-infra | 01:10 | |
| *** ashleighfarnham has quit IRC | 01:11 | |
| *** ashleighfarnham has joined #openstack-infra | 01:11 | |
| mordred | yah. we're more working on the general problem of "how do jobs publish things" | 01:11 |
| mordred | hoping to solve it in such as way as that you can easily not care how it works ever | 01:12 |
| jeblair | cody-somerville: many of the breadcrumbs seem too verbose to be useful... i wonder if screens should have a 'short title'...? | 01:13 |
| jeblair | cody-somerville: so rather than "All changes in My ..." we would have "My changes". | 01:14 |
| mordred | jeblair: is this a gertty change? | 01:14 |
| *** Daisy has quit IRC | 01:14 | |
| jeblair | mordred: yeah, the one cody-somerville linked to earlier: https://review.openstack.org/288250 | 01:15 |
| *** Daisy has joined #openstack-infra | 01:15 | |
| cody-somerville | jeblair: The same idea occurred to me as well. | 01:15 |
| mordred | Black right pointing small triangle | 01:15 |
| *** ashleighfarnham has quit IRC | 01:15 | |
| mordred | that's such a better letter name than "epsilon" | 01:15 |
| jeblair | mordred: ya, i didn't even know you could do that.... << cody-somerville :) | 01:15 |
| *** bpokorny_ has joined #openstack-infra | 01:15 | |
| *** Daisy has quit IRC | 01:16 | |
| *** Daisy has joined #openstack-infra | 01:16 | |
| jeblair | (might be a good followup change to experiment with that) | 01:17 |
| *** bpokorny has quit IRC | 01:18 | |
| *** bpokorny_ has quit IRC | 01:20 | |
| *** Sukhdev has joined #openstack-infra | 01:21 | |
| bkero | → | 01:22 |
| *** pvaneck has quit IRC | 01:22 | |
| cody-somerville | jeblair: Yup. Was also thinking about making the breadcrumb interactive in subsequent change - e.g. click breadcrumb to switch to that view | 01:24 |
| cody-somerville | jeblair: or be able to pan when there is overflow. might be able to do something like show more of the title if selected or something. | 01:25 |
| *** ddieterly has joined #openstack-infra | 01:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Add zlib1g-dev to bindep-fallback.txt https://review.openstack.org/288805 | 01:28 |
| jeblair | cody-somerville: yeah, clicking seems like a good next step | 01:28 |
| *** davide85_ has quit IRC | 01:29 | |
| *** Daisy has quit IRC | 01:30 | |
| *** mtanino has joined #openstack-infra | 01:30 | |
| *** dims has joined #openstack-infra | 01:30 | |
| *** Daisy has joined #openstack-infra | 01:30 | |
| cody-somerville | jeblair: fyi, in my testing I noticed that I could not get the TOP_SCREEN command to trigger. If I'm reading right, it should be alt+home key. I think it might be related to Unity binding the meta key. I did change the binding locally to test but figured I'd point it out since Ubuntu is popular. | 01:30 |
| jeblair | cody-somerville: ah, thanks. indeed that's what it's supposed to be (wfm under xfce), and i can understand not having that would make navigation slightly cumbersome... i'm open to alternate suggestions. | 01:32 |
| jeblair | cody-somerville: er, wait, i think it's ctrl-home | 01:32 |
| jeblair | cody-somerville: oh weird | 01:33 |
| fungi | i find it interesting that just home and end don't do top/bottom of screen | 01:33 |
| cody-somerville | jeblair: Says meta home. In any case, I tried all three keys down there with home and it didn't work. But changing it to say ctrl + p did. | 01:33 |
| jeblair | cody-somerville: it does say 'meta home' and when i hit 'ctrl home' it triggers it. | 01:33 |
| jeblair | i can not immediately explain that. | 01:34 |
| jeblair | fungi: yeah, they should, which is why i didn't bind top_screen to home... this is probably fixable. | 01:34 |
| cody-somerville | jeblair: so yea, a second default keybinding might not be a bad idea. | 01:34 |
| *** Daisy has quit IRC | 01:34 | |
| cody-somerville | home seems to be bound to CURSOR_MAX_LEFT | 01:35 |
| cody-somerville | which does seem to work as expected when you're entering text. | 01:35 |
| cody-somerville | We could definitely have it do top/bottom of screen when not entering text. | 01:36 |
| jeblair | ++ | 01:36 |
| fungi | that would make sense to my finger memory | 01:37 |
| cody-somerville | (and we should find more excuses to add more unicode symbols, they're fun! lol) | 01:37 |
| clarkb | really need to make nodepool builders reload their config | 01:39 |
| *** yamahata has quit IRC | 01:39 | |
| *** mtanino has quit IRC | 01:40 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Only create 1024MB swap in tripleo-ci https://review.openstack.org/288827 | 01:42 |
| *** sdake has quit IRC | 01:43 | |
| clarkb | waiting on the ovh bhs image upload to complete so I can restart the builder service | 01:43 |
| *** rfolco has joined #openstack-infra | 01:44 | |
| *** Sukhdev has quit IRC | 01:45 | |
| *** markvoelker has joined #openstack-infra | 01:46 | |
| *** alivigni76 has joined #openstack-infra | 01:48 | |
| fungi | oh, also if mordred's nodepool debug logging patches landed, might also be a good time for a nodepoold restart | 01:49 |
| *** alivigni76 is now known as _ari_ | 01:49 | |
| fungi | speaking of ovh | 01:49 |
| *** _ari_ has quit IRC | 01:49 | |
| *** rfolco has quit IRC | 01:52 | |
| *** keedya has quit IRC | 01:54 | |
| clarkb | I am tempted to stop the builder anyways | 01:55 |
| clarkb | since we need a new image in bhs for zlib | 01:55 |
| *** dims has quit IRC | 01:55 | |
| NobodyCam | yep error has changed to : ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting | 01:55 |
| *** keedya has joined #openstack-infra | 01:56 | |
| clarkb | NobodyCam: jpg is present on the image snow except for in ovh-bhs1 | 01:56 |
| fungi | clarkb: i suppose if ovh is taking too long to update, given the other clouds we have in production we could just delete all ubuntu-trusty images there and let it not build new instances of that until it gets a new upload | 01:56 |
| clarkb | fungi: I imagine it owuld help a lot if the disk image was half the size | 01:57 |
| fungi | yeah, that too | 01:57 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Revert "Use swapfile environment in CI" https://review.openstack.org/288716 | 01:58 |
| clarkb | osic images are building now | 01:58 |
| clarkb | where building == uploading | 01:58 |
| *** kzaitsev_mb has quit IRC | 01:58 | |
| clarkb | jenkins02 is still going :| | 01:59 |
| *** tiswanso has joined #openstack-infra | 01:59 | |
| *** tiswanso has quit IRC | 01:59 | |
| *** tiswanso has joined #openstack-infra | 02:00 | |
| cody-somerville | jeblair: any thoughts on underlining corresponding letter to keybinding in buttons? http://imagebin.ca/v/2Z9SPVqJbA0M | 02:00 |
| nibalizer | cody-somerville: i like that | 02:02 |
| clarkb | I just kicked off another ubuntu trusty build for zlib | 02:04 |
| *** HenryG has quit IRC | 02:06 | |
| fungi | clarkb: i often resort to putting jenkins02 into shutdown mode to get it to update faster/with fewer api errors | 02:06 |
| *** HenryG has joined #openstack-infra | 02:07 | |
| clarkb | 2/4 images uploaded to osic | 02:08 |
| clarkb | no slaves yet | 02:08 |
| *** superdan is now known as dansmith | 02:09 | |
| *** apoorvad has quit IRC | 02:12 | |
| clarkb | quick someone generate some load | 02:14 |
| fungi | need me to burn out a ton of ubuntu-trusty ready nodes? i can pretty trivially with gertty mass reviews | 02:18 |
| clarkb | fungi: yes please :) | 02:18 |
| clarkb | I mostly just want to boot a VM there and demand is too low with the ratios | 02:19 |
| clarkb | all 4 images are in osic now | 02:19 |
| clarkb | the dib image build for ubuntu-trusty to add zlib is in progress | 02:19 |
| clarkb | (we will want ot queue up reuploads for that image once it is done | 02:19 |
| clarkb | and jenkinses are jjbing | 02:20 |
| fungi | i went lightly, but should be quite a few launching in moments | 02:20 |
| clarkb | yup there is one now thank you | 02:20 |
| clarkb | the trick here is going to be did we get the floating ip pool stuff set properly (I believe so) | 02:21 |
| *** asselin__ has joined #openstack-infra | 02:21 | |
| clarkb | jenkins02 is into xstatic | 02:21 |
| *** jcooley_ has quit IRC | 02:21 | |
| clarkb | I think it may have 502'd earlier but instead of dealing with it immediately I think we do 04 and 05 then come back to 02 | 02:22 |
| clarkb | since 02 is mostly up to date and slow as molasses | 02:22 |
| clarkb | and its ready \o/ | 02:22 |
| fungi | i would shutdown 02, clean up and restart it once it's idle, and update jjb with it offline | 02:22 |
| * clarkb does a happy dance and waits for a job to start on it | 02:22 | |
| clarkb | ready means that fips and all that work | 02:22 |
| fungi | 02 has a tendency to get pretty awful (worse than 01 for some reason) | 02:23 |
| clarkb | fungi: https://jenkins03.openstack.org/job/gate-python-cinderclient-requirements-bindep/1/console | 02:23 |
| clarkb | note that the osic flavor is 4 vcpu instead of 8, but testing tempest we still run super fast | 02:24 |
| clarkb | and that job succeeded | 02:24 |
| *** apoorvad has joined #openstack-infra | 02:24 | |
| clarkb | and the node is deleted | 02:25 |
| clarkb | we have successfully clouded | 02:25 |
| *** _ari_ has joined #openstack-infra | 02:26 | |
| clarkb | 02 just stopped updating I am going to fg 04 | 02:26 |
| clarkb | and leave 02 for later | 02:26 |
| *** _ari_ is now known as ari | 02:29 | |
| clarkb | https://jenkins07.openstack.org/job/glance-coverage-bindep/2/console | 02:29 |
| *** ari is now known as _ari_ | 02:29 | |
| clarkb | fungi: that job failed due to bad config in glance but it also said there was no mysql db | 02:32 |
| clarkb | fungi: which would be a regression on the unittests if that is consisten | 02:32 |
| *** _ari_ has quit IRC | 02:33 | |
| cody-somerville | Wow. I just realized the primary developer/maintainer of urwid lives here in Ottawa. | 02:34 |
| fungi | clarkb: indeed, the experimental coverage job may not set up the databases. i should check the voting version | 02:35 |
| *** |-paul-| has quit IRC | 02:36 | |
| fungi | clarkb: hrm, they both call the mysql-prep and mysql-db macros | 02:36 |
| fungi | clarkb: and you can see the sudo mysql commands happening in the console log on that job too | 02:37 |
| *** baoli has joined #openstack-infra | 02:38 | |
| fungi | clarkb: are you talking about the "NOTICE: database "openstack_citest" does not exist, skipping DROP DATABASE" message (note that it's wrapped to the next line)? | 02:39 |
| *** _ari_ has joined #openstack-infra | 02:39 | |
| fungi | clarkb: the failure looks like it's a discover issue... glance's normal coverage job may have always been broken | 02:39 |
| fungi | Non-zero exit code (1) from test listing. | 02:39 |
| fungi | which i think may be precipitated by the line above that | 02:40 |
| fungi | Unrecognized option '[report] ignore-errors=' in config file .coveragerc | 02:40 |
| *** sbalukoff has quit IRC | 02:41 | |
| fungi | not to mention this is coverage for glance on stable/kilo | 02:41 |
| fungi | so it's also probable nobody cares that job's been broken | 02:42 |
| *** thorst has quit IRC | 02:42 | |
| *** thorst has joined #openstack-infra | 02:43 | |
| fungi | as for the "errors" from the database stup macros, they attempt to drop the database they're going to create in case it already exists, for the sake of keeping idempotent | 02:43 |
| fungi | and also to make these versions of the jobs safe to run on the bare-trusty instances if we'd ended up taking that route temporarily | 02:44 |
| *** _ari_ has quit IRC | 02:46 | |
| *** tqtran has quit IRC | 02:46 | |
| *** _ari_ has joined #openstack-infra | 02:46 | |
| *** woodster_ has quit IRC | 02:47 | |
| *** thorst has quit IRC | 02:51 | |
| *** _ari_ has quit IRC | 02:52 | |
| *** _ari_ has joined #openstack-infra | 02:53 | |
| *** _ari_ has quit IRC | 02:53 | |
| *** _ari_ has joined #openstack-infra | 02:54 | |
| *** _ari_ has quit IRC | 02:55 | |
| *** _ari_ has joined #openstack-infra | 02:55 | |
| *** _ari_ has quit IRC | 02:55 | |
| *** apoorvad has quit IRC | 02:55 | |
| *** _ari_ has joined #openstack-infra | 02:56 | |
| *** noorul has joined #openstack-infra | 02:59 | |
| *** _ari_ has quit IRC | 03:01 | |
| *** _ari_ has joined #openstack-infra | 03:01 | |
| *** bpokorny has joined #openstack-infra | 03:02 | |
| openstackgerrit | Merged openstack/requirements: Need a new minimum for oslo.cache in Mitaka https://review.openstack.org/288703 | 03:02 |
| *** Jeffrey4l has joined #openstack-infra | 03:03 | |
| clarkb | ah | 03:04 |
| fungi | those experimental bindep jobs were a carpet bombing run. we only analyzed the results of ones who failed when their non-bindep counterparts passed | 03:06 |
| nibalizer | clarkb: fungi would you be willing to 2a this? https://review.openstack.org/#/c/288706/ | 03:06 |
| fungi | thought i already had | 03:07 |
| fungi | approved now | 03:07 |
| clarkb | I just kicked off some uploads now that the ubuntu-trusty image is rebuilt with zlib things | 03:07 |
| fungi | looks like it doesn't get applied anywhere that /etc/logstash would be created for other purposes | 03:07 |
| *** noorul has quit IRC | 03:08 | |
| clarkb | I am not doing rax because thats not super reliable nayways | 03:09 |
| *** gildub has joined #openstack-infra | 03:09 | |
| clarkb | fungi: if we are brae we can set the max servers to 100 and go weekend :) | 03:10 |
| *** ddieterly has quit IRC | 03:14 | |
| *** baoli has quit IRC | 03:16 | |
| openstackgerrit | Merged openstack-infra/puppet-subunit2sql: Switch /etc/logstash to /etc/subunit2sql https://review.openstack.org/288706 | 03:17 |
| fungi | clarkb: well, the next phase is to scour project-config for any job configs still mentioning bare-trusty and update them (and figure out which are inheriting bare-trusty from projects.yaml) and then start switching all the devstack-trusty jobs over to bare-trusty | 03:18 |
| clarkb | yup | 03:19 |
| clarkb | 04 finished but errored so I have restarted its jjb updates | 03:19 |
| fungi | oh, you mean max-servers on osic | 03:19 |
| fungi | i was thinking min-ready on ubuntu-trusty for some reason | 03:19 |
| clarkb | yes that :) | 03:19 |
| clarkb | but we can do both | 03:19 |
| fungi | which we're nowhere near ready to need that many on hand | 03:19 |
| fungi | but yeah, osic sounds good. i'll be around all weekend | 03:20 |
| clarkb | I mostly want the ratio higher so that the chances of getting an ubuntu-trusty-2-node pair are higher which can be used to say the 2-node setup works well enough there | 03:22 |
| fungi | yeah, that would be great | 03:22 |
| clarkb | then we can add devstack-trusty-2-node | 03:22 |
| clarkb | so far it looks good and no leaked fips | 03:22 |
| clarkb | also ubuntu-trusty up to date everywhere but rax, ovh, and internap | 03:24 |
| mtreinish | fungi: oh, reading the backscroll about the coverage jobs, do you think it's worth adding the post pipeline to subunit2sql collection? | 03:26 |
| clarkb | it probably won't hurt | 03:27 |
| mtreinish | granted if it's failing before there's any subunit output that means it won't show up on openstack-health, but I forgot about those jobs | 03:27 |
| clarkb | though only coverage generates subunit there | 03:27 |
| mtreinish | true | 03:27 |
| clarkb | fungi: maybe tomorrow morning we can bump osic and make sure it stays happy? | 03:27 |
| clarkb | I am running out of steam and cooking dinner | 03:27 |
| clarkb | 04 is done, starting 05 | 03:28 |
| clarkb | I should probably figure out this grafyaml thing and add an osic | 03:29 |
| nibalizer | fungi: thanks! | 03:31 |
| mtreinish | nibalizer: hmm, I'm surprised it didn't pull in the patch: http://puppetdb.openstack.org/report/subunit-worker02.openstack.org/70be193085e675cc358d8cba5d29cce944d4cac2 | 03:32 |
| mtreinish | it ran after it merged | 03:32 |
| *** mtanino has joined #openstack-infra | 03:32 | |
| nibalizer | it can take up to 15 minutes | 03:33 |
| nibalizer | so next run will definitely have it | 03:33 |
| mtreinish | nibalizer: ah, ok | 03:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Grafana for OSIC https://review.openstack.org/288832 | 03:37 |
| *** fawadkhaliq has quit IRC | 03:37 | |
| fungi | clarkb: sounds great | 03:40 |
| fungi | have a good evening | 03:40 |
| clarkb | internap has up to date ubuntu-trsuty | 03:41 |
| clarkb | so just ovh left and they are uploading | 03:41 |
| clarkb | fungi: I am going to put 02 in shutdown mode now | 03:41 |
| *** esikachev has joined #openstack-infra | 03:42 | |
| fungi | that's often the only way i can complete a large update on it | 03:42 |
| *** esikachev has quit IRC | 03:46 | |
| *** thorst has joined #openstack-infra | 03:49 | |
| *** HenryG has quit IRC | 03:54 | |
| *** thorst has quit IRC | 03:56 | |
| *** yamamoto has joined #openstack-infra | 03:58 | |
| *** yamamoto has quit IRC | 04:01 | |
| *** tiswanso has quit IRC | 04:08 | |
| *** kingia_ has joined #openstack-infra | 04:20 | |
| *** yamamoto has joined #openstack-infra | 04:21 | |
| *** HenryG has joined #openstack-infra | 04:22 | |
| *** jtomasek has joined #openstack-infra | 04:23 | |
| *** kingia has quit IRC | 04:23 | |
| *** Daisy has joined #openstack-infra | 04:31 | |
| *** yamamoto has quit IRC | 04:31 | |
| *** claudiub|2 has joined #openstack-infra | 04:34 | |
| *** dimtruck is now known as zz_dimtruck | 04:34 | |
| *** Daisy has quit IRC | 04:35 | |
| *** claudiub has quit IRC | 04:37 | |
| *** zz_dimtruck is now known as dimtruck | 04:41 | |
| *** treaki_ has joined #openstack-infra | 04:43 | |
| *** treaki has quit IRC | 04:47 | |
| *** roxanaghe has joined #openstack-infra | 04:51 | |
| *** dimtruck is now known as zz_dimtruck | 04:52 | |
| *** thorst has joined #openstack-infra | 04:54 | |
| *** Keedya_86 has joined #openstack-infra | 04:58 | |
| *** bpokorny has quit IRC | 04:59 | |
| *** Keedya_86 has quit IRC | 04:59 | |
| *** thorst has quit IRC | 05:01 | |
| *** keedya has quit IRC | 05:02 | |
| *** HenryG has quit IRC | 05:11 | |
| *** HenryG has joined #openstack-infra | 05:13 | |
| *** HenryG has quit IRC | 05:14 | |
| *** claudiub|2 has quit IRC | 05:14 | |
| *** HenryG has joined #openstack-infra | 05:14 | |
| *** Jeffrey4l has quit IRC | 05:17 | |
| *** yamamoto has joined #openstack-infra | 05:32 | |
| *** zz_dimtruck is now known as dimtruck | 05:34 | |
| *** yamamoto has quit IRC | 05:38 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: (WIP) Add functional tests https://review.openstack.org/288840 | 05:40 |
| *** hdd has joined #openstack-infra | 05:41 | |
| *** esikachev has joined #openstack-infra | 05:42 | |
| *** dimtruck is now known as zz_dimtruck | 05:44 | |
| *** esikachev has quit IRC | 05:47 | |
| *** salv-orlando has joined #openstack-infra | 05:52 | |
| *** ashleighfarnham has joined #openstack-infra | 05:53 | |
| *** salv-orl_ has quit IRC | 05:55 | |
| *** vivekd has joined #openstack-infra | 05:59 | |
| *** thorst has joined #openstack-infra | 05:59 | |
| *** thorst has quit IRC | 06:06 | |
| *** harlowja_at_home has quit IRC | 06:07 | |
| *** hdd has quit IRC | 06:19 | |
| clarkb | ok 05 is all done and I think 02 is finally happy but rerunning one more time there to see it noop | 06:20 |
| clarkb | I probably won't stay awake to watch it | 06:20 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/288847 | 06:20 |
| *** Daisy_ has joined #openstack-infra | 06:20 | |
| *** HenryG has quit IRC | 06:20 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 06:29 |
| *** vivekd has quit IRC | 06:30 | |
| *** ashleighfarnham has quit IRC | 06:51 | |
| *** fawadkhaliq has joined #openstack-infra | 06:54 | |
| *** yamahata has joined #openstack-infra | 06:54 | |
| *** fawadkhaliq has quit IRC | 06:56 | |
| *** roxanaghe has quit IRC | 07:00 | |
| *** Daisy_ has quit IRC | 07:02 | |
| *** thorst has joined #openstack-infra | 07:05 | |
| *** vivekd has joined #openstack-infra | 07:05 | |
| *** thorst has quit IRC | 07:11 | |
| *** zz_dimtruck is now known as dimtruck | 07:13 | |
| *** Daisy has joined #openstack-infra | 07:23 | |
| *** Daisy has quit IRC | 07:27 | |
| *** vivekd has quit IRC | 07:40 | |
| *** esikachev has joined #openstack-infra | 07:43 | |
| *** esikachev has quit IRC | 07:47 | |
| *** sputnik13 has quit IRC | 07:49 | |
| *** roxanaghe has joined #openstack-infra | 08:01 | |
| *** roxanaghe has quit IRC | 08:06 | |
| *** thorst has joined #openstack-infra | 08:09 | |
| *** thorst has quit IRC | 08:16 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 08:17 |
| AJaeger | infra-root, dhellmann, ttx: Requirements repo is broken, I expect that the change above will fix it. Please merge it to unblock requirements repo | 08:19 |
| *** HeOS has quit IRC | 08:19 | |
| AJaeger | infra-root, on http://logs.openstack.org/47/288847/1/gate/gate-bindep-fallback-ubuntu-trusty/71b6937/console.html I got: | 08:20 |
| AJaeger | "Unable to connect to mirror.bhs1.ovh.openstack.org:http:" | 08:20 |
| AJaeger | Is ovh broken? | 08:20 |
| AJaeger | infra-root, ovh-bhs1 seems to have broken networking, I see lots of image failures there | 08:26 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Disable ovh-bhs1 https://review.openstack.org/288859 | 08:28 |
| AJaeger | infra-root, a band-aid suggestion: disable ovh-bhs1 ^ | 08:28 |
| *** [1]Thelo has joined #openstack-infra | 08:29 | |
| *** asselin__ has quit IRC | 08:30 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 08:31 |
| *** Thelo has quit IRC | 08:32 | |
| *** [1]Thelo is now known as Thelo | 08:32 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Convert storlets, trove, zuul jobs to ubuntu-trusty https://review.openstack.org/288860 | 08:35 |
| *** achanda has quit IRC | 08:36 | |
| *** achanda has joined #openstack-infra | 08:40 | |
| *** esikachev has joined #openstack-infra | 08:42 | |
| *** esikachev has quit IRC | 08:46 | |
| *** achanda has quit IRC | 08:50 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 08:54 |
| *** sbalukoff has joined #openstack-infra | 09:00 | |
| *** thorst has joined #openstack-infra | 09:14 | |
| *** thorst has quit IRC | 09:20 | |
| *** dimtruck is now known as zz_dimtruck | 09:22 | |
| *** HeOS has joined #openstack-infra | 09:24 | |
| *** roxanaghe has joined #openstack-infra | 09:30 | |
| *** roxanaghe has quit IRC | 09:34 | |
| *** rguillebert has joined #openstack-infra | 10:05 | |
| *** e0ne has joined #openstack-infra | 10:05 | |
| *** yamahata has quit IRC | 10:06 | |
| *** e0ne has quit IRC | 10:08 | |
| *** yamamoto has joined #openstack-infra | 10:09 | |
| *** vivekd has joined #openstack-infra | 10:11 | |
| *** yamamoto has quit IRC | 10:13 | |
| *** thorst has joined #openstack-infra | 10:18 | |
| *** thorst has quit IRC | 10:26 | |
| *** esikachev has joined #openstack-infra | 10:27 | |
| *** yamamoto has joined #openstack-infra | 10:28 | |
| *** yamamoto has quit IRC | 10:31 | |
| *** yamamoto has joined #openstack-infra | 10:35 | |
| *** yamamoto has quit IRC | 10:35 | |
| *** vivekd has quit IRC | 10:42 | |
| *** HeOS has quit IRC | 10:45 | |
| *** [HeOS] has joined #openstack-infra | 10:45 | |
| *** esikachev has quit IRC | 10:49 | |
| *** claudiub|2 has joined #openstack-infra | 10:53 | |
| *** [HeOS] has quit IRC | 10:54 | |
| *** HeOS has joined #openstack-infra | 10:56 | |
| *** aspiers has quit IRC | 10:56 | |
| *** aspiers has joined #openstack-infra | 10:57 | |
| *** HeOS has quit IRC | 11:03 | |
| *** HeOS has joined #openstack-infra | 11:06 | |
| *** ogelbukh has quit IRC | 11:07 | |
| *** claudiub|2 has quit IRC | 11:15 | |
| *** roxanaghe has joined #openstack-infra | 11:18 | |
| *** roxanaghe has quit IRC | 11:23 | |
| *** thorst has joined #openstack-infra | 11:23 | |
| *** thorst has quit IRC | 11:30 | |
| *** pcaruana has quit IRC | 11:34 | |
| *** yamamoto has joined #openstack-infra | 11:36 | |
| *** yamamoto has quit IRC | 11:43 | |
| *** Jeffrey4l has joined #openstack-infra | 11:47 | |
| *** ddieterly has joined #openstack-infra | 11:49 | |
| *** salv-orl_ has joined #openstack-infra | 11:52 | |
| *** salv-orlando has quit IRC | 11:55 | |
| *** Daisy has joined #openstack-infra | 11:57 | |
| *** claudiub|2 has joined #openstack-infra | 12:01 | |
| *** HeOS has quit IRC | 12:03 | |
| *** dims has joined #openstack-infra | 12:11 | |
| *** dims has quit IRC | 12:13 | |
| *** lhinds has joined #openstack-infra | 12:14 | |
| lhinds | hi, would openstack-infra be a good place to ask about an issue with the wiki (OpenID) issue? | 12:14 |
| *** ddieterly is now known as ddieterly[away] | 12:17 | |
| *** ddieterly[away] is now known as ddieterly | 12:17 | |
| *** dkehn has quit IRC | 12:19 | |
| *** gildub has quit IRC | 12:23 | |
| *** thorst has joined #openstack-infra | 12:28 | |
| *** thorst has quit IRC | 12:36 | |
| *** ddieterly has quit IRC | 12:49 | |
| AJaeger | lhinds: in general yes - but not right now. Best come back Monday after 15:00 UTC | 12:54 |
| *** zeih has joined #openstack-infra | 12:56 | |
| *** thorst has joined #openstack-infra | 12:57 | |
| *** thorst has quit IRC | 13:02 | |
| *** thorst has joined #openstack-infra | 13:03 | |
| *** roxanaghe has joined #openstack-infra | 13:06 | |
| *** ihrachys has joined #openstack-infra | 13:07 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 13:08 |
| *** dims has joined #openstack-infra | 13:08 | |
| dims | AJaeger : hey | 13:10 |
| dims | AJaeger : looks like https://review.openstack.org/#/c/288858/ is a rush? | 13:10 |
| openstackgerrit | Merged openstack-infra/project-config: Sort bindep-fallback.txt https://review.openstack.org/288810 | 13:10 |
| AJaeger | dims: yes. Either we remove bindep-fallback or fix it ;) | 13:11 |
| *** thorst has quit IRC | 13:11 | |
| *** roxanaghe has quit IRC | 13:11 | |
| AJaeger | dims, we switched from bare-trusty - which ignored other-requirements - to ubuntu-trusty which uses it | 13:11 |
| AJaeger | (I meant other-requirements above not bindep-fallback) | 13:11 |
| dims | +2A | 13:11 |
| AJaeger | thanks, dims. Hope it now includes all dependencies... | 13:12 |
| AJaeger | infra-root, seems that ovs-bhs1 is ok again... | 13:13 |
| dims | thanks for working through it AJaeger | 13:13 |
| AJaeger | you're welcome | 13:14 |
| *** yamamoto has joined #openstack-infra | 13:15 | |
| *** yamamoto has quit IRC | 13:15 | |
| *** yamamoto has joined #openstack-infra | 13:15 | |
| *** openstackgerrit_ has quit IRC | 13:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 13:17 |
| AJaeger | dims: Just updated again - saw pypy failing as well... | 13:17 |
| dims | AJaeger : no worries | 13:18 |
| *** openstackgerrit_ has joined #openstack-infra | 13:18 | |
| AJaeger | let's see whether it passes. | 13:18 |
| *** Daisy has quit IRC | 13:18 | |
| * AJaeger will be back later | 13:18 | |
| dims | AJaeger : y will watch it | 13:21 |
| *** zeih has quit IRC | 13:23 | |
| *** rfolco has joined #openstack-infra | 13:23 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Add heat-translator>=0.4.0 https://review.openstack.org/287581 | 13:26 |
| pabelanger | AJaeger: Hows bindep jobs looking this morning? I think clarkb and fungi got the latest fixes merged but haven't researched any failures in the gate | 13:27 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Bump python-neutronclient to 4.1.1 https://review.openstack.org/288428 | 13:27 |
| *** ddieterly has joined #openstack-infra | 13:27 | |
| *** rfolco has quit IRC | 13:29 | |
| *** amotoki has joined #openstack-infra | 13:30 | |
| *** ddieterly has quit IRC | 13:31 | |
| *** daemontool_ has quit IRC | 13:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Clean up bindep-fallback.txt https://review.openstack.org/288811 | 13:34 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/288847 | 13:35 |
| *** hdd has joined #openstack-infra | 13:37 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Remove experimental bindep infra jobs https://review.openstack.org/288868 | 13:39 |
| *** yamamoto has quit IRC | 13:43 | |
| openstackgerrit | Francesco Longo proposed openstack-infra/project-config: Added IoTronic project. https://review.openstack.org/286113 | 13:49 |
| *** dims has quit IRC | 13:53 | |
| AJaeger | pabelanger: requirements repo had a broken other-requirements file, hope it's fixed now. | 13:54 |
| AJaeger | pabelanger: otherwise no bindep related problems that showed up | 13:54 |
| AJaeger | pabelanger: https://review.openstack.org/#/c/288858/ - that one needs to be enhanced for CentOS at one point... | 13:55 |
| pabelanger | AJaeger: Ya, we'll need to get centos-7 dib going soon, and come up with a way for projects to validate bindep on centos (even fedora) | 13:58 |
| AJaeger | pabelanger: remember to test on requirements repo - or better: On each repo that has other-requirements file | 13:58 |
| *** hdd has quit IRC | 13:59 | |
| AJaeger | pabelanger: if you want to review some more moves to ubuntu-trusty, please check https://review.openstack.org/#/q/topic:bindep+status:open | 13:59 |
| AJaeger | pabelanger: but if people just recheck without checking what's broken (some requirements jobs had 3+ rechecks) and fixing it or asking how to fix it, we will not learn about it ;( | 14:02 |
| pabelanger | AJaeger: indeed. I've been scanning status.o.o/zuul trying to catch failures | 14:03 |
| AJaeger | this morning ovh-bhs1 was completely broken, every job failed there since the mirror couldn't be reached ;( | 14:04 |
| AJaeger | That "helped" a lot in finding and fixing issues ;( | 14:04 |
| *** chlong_ has quit IRC | 14:05 | |
| *** hdd has joined #openstack-infra | 14:08 | |
| *** thorst has joined #openstack-infra | 14:08 | |
| *** hdd has quit IRC | 14:09 | |
| *** mtanino has quit IRC | 14:13 | |
| *** claudiub has joined #openstack-infra | 14:14 | |
| *** HenryG has joined #openstack-infra | 14:15 | |
| *** claudiub|2 has quit IRC | 14:15 | |
| *** nihilifer has quit IRC | 14:15 | |
| *** thorst has quit IRC | 14:15 | |
| *** nihilifer has joined #openstack-infra | 14:17 | |
| *** Daisy has joined #openstack-infra | 14:19 | |
| *** openstackgerrit_ has quit IRC | 14:20 | |
| *** gnuoy_ has joined #openstack-infra | 14:21 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:21 | |
| *** roxanaghe has joined #openstack-infra | 14:22 | |
| *** Daisy has quit IRC | 14:23 | |
| *** dkehn has joined #openstack-infra | 14:25 | |
| *** gnuoy_ has quit IRC | 14:26 | |
| *** asselin__ has joined #openstack-infra | 14:27 | |
| *** roxanaghe has quit IRC | 14:27 | |
| *** asselin__ has quit IRC | 14:31 | |
| *** openstackgerrit_ has quit IRC | 14:32 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:33 | |
| *** yamamoto has joined #openstack-infra | 14:44 | |
| *** dims has joined #openstack-infra | 14:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: ubuntu-trusty: misc,openstack-planet,... https://review.openstack.org/288874 | 14:49 |
| *** sdake has joined #openstack-infra | 14:51 | |
| *** yamamoto has quit IRC | 14:52 | |
| *** sdake_ has joined #openstack-infra | 14:56 | |
| *** sdake has quit IRC | 14:57 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Convert devstac* to ubuntu-trusty https://review.openstack.org/288877 | 15:08 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Convert elastic-recheck to ubuntu-trusty https://review.openstack.org/288878 | 15:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Convert grenade to use ubuntu-trusty https://review.openstack.org/288879 | 15:12 |
| fungi | lhinds: i can imagine there's one significant issue you'll encounter with openid authentication to the wiki. if you're trying to create a new account (don't have an existing account on the wiki or switching to a different account which didn't exist before) then that won't work just now. we've got new account creation disabled while we work through a serious spam problem there | 15:12 |
| AJaeger | fungi, thanks for approving 288858 - should merge in 4 mins since dims was faster ;) | 15:13 |
| openstackgerrit | Merged openstack/requirements: Fix other-requirements https://review.openstack.org/288858 | 15:13 |
| AJaeger | that was 4 seconds instead of 4 mins ;) | 15:13 |
| *** hdd has joined #openstack-infra | 15:13 | |
| *** thorst has joined #openstack-infra | 15:13 | |
| fungi | AJaeger: yeah, i missed the other approval on it when i skimmed, saw it after i approved | 15:14 |
| fungi | good catch | 15:14 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Convert job-builder to use ubuntu-trusty https://review.openstack.org/288881 | 15:14 |
| *** dizquierdo has joined #openstack-infra | 15:15 | |
| fungi | AJaeger: also looks like some requirements changes in the gate passed behind that, so should be good | 15:15 |
| AJaeger | fungi, pabelanger and myself have some changes for ubuntu-trusty conversion and cleanup up - nothing urgent. https://review.openstack.org/#/q/topic:bindep+status:open is complete list | 15:15 |
| AJaeger | fungi: yes, requirements looks fine. That other-requirements file was never tested ;( | 15:16 |
| fungi | for some reason i thought someone had tested out all the repos with other-requirements.txt files in them but now i don't recall who (and know it wasn't me) | 15:16 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/release-tools: Do not merge - TESTING https://review.openstack.org/288882 | 15:18 |
| fungi | and also your database package additions there remind me that i'll want to start thinking about maybe making the database setup job macros no-op when the executables they call aren't in path | 15:18 |
| AJaeger | I think we had added jobs for it but might we forgot them somehow ;( | 15:18 |
| AJaeger | fungi: good idea... | 15:18 |
| fungi | that would allow projects who don't need database support for any of their project-specific jobs to share the same job templates as those who do, but easily omit them from other-requirements.txt | 15:19 |
| * AJaeger tests now the repos with other-requirements ... | 15:19 | |
| fungi | the down-side to making that autodiscovered is that the database tests in many jobs are also opportunistic, so a bug preventing installation of database tools could go silently undetected and those projects would just stop running their opportunistic database tests | 15:20 |
| *** thorst has quit IRC | 15:21 | |
| fungi | another option would be to have a databases option to the install-distro-packages macro which appends the required packages to the other-requirements.txt before feedint it to bindep, but that way leaves a hole where a project not using the databases can still quietly grow a dependency on a package which one of the database packages depends on | 15:21 |
| openstackgerrit | Merged openstack/requirements: Add heat-translator>=0.4.0 https://review.openstack.org/287581 | 15:22 |
| AJaeger | for now I would rather get rid of firefox and other odd bits from bindep-fallback | 15:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: DO NOT MERGE - just testing https://review.openstack.org/288883 | 15:23 |
| *** sdake_ is now known as sdake | 15:24 | |
| fungi | yep, longer term i'd love for projects which need special database setup to run some of their tests start doing it explicitly (maybe moving the tooling to do the setup into their repo if we can standardize on what that script would be named so it can be consistently called from jobs) | 15:25 |
| fungi | but also the fallback list does provide us with opportunities for some low-hanging fruit, like getting horizon an other-projects.txt tested out so we can drop web browsers from the fallback list ;) | 15:25 |
| AJaeger | yep | 15:26 |
| fungi | up-side to this design is that projects wanting to add a replacement package list are able to self-test that | 15:26 |
| AJaeger | what about branches? our fallback list is branch independent, so horizon needs to add other-projects for stable branches as well, doesn't it? | 15:27 |
| fungi | also wanting to augment devstack to use bindep rather than its current home-grown package lists (maybe teh devstack requirements can even move into individual repos as well that way) | 15:27 |
| fungi | AJaeger: if we're going to remove an entry from the fallback list, yes it needs to first be set up for all branches of whatever was consuming that | 15:27 |
| fungi | but again, the per-project other-requirements.txt additions are self-testing, just need to remember to add them to all branches which need one | 15:28 |
| openstackgerrit | Merged openstack/requirements: Bump python-neutronclient to 4.1.1 https://review.openstack.org/288428 | 15:28 |
| AJaeger | fungi, so next we should document and announce some of this. What do you think of writing an announcement email to openstack-dev? | 15:29 |
| fungi | it's removals from the fallback list which aren't tested, because figuring out what might break and automatically running jobs for that is a mostly intractible problem | 15:29 |
| * AJaeger saw some requirements job rechecked 3+ times that failed because of ubuntu-trusty... | 15:29 | |
| fungi | AJaeger: yeah, i planned to write up an announcement once the bulk of common jobs are off bare-trusty and not broken, so probably monday | 15:29 |
| AJaeger | fungi, thanks | 15:29 |
| AJaeger | fungi, do we have documentation for the bindep work? If not, where should it live? | 15:30 |
| fungi | AJaeger: rechecks from failures because of the broken package name in the requirements repo's other-requirements.txt file, or for other failures on ubuntu-trusty? | 15:30 |
| AJaeger | because of broken package names in requirements file - so they either never looked at log files or didn't understand failure | 15:31 |
| fungi | AJaeger: we have documentation published for bindep, but could stand to have some simple details somewhere (i haven't thought about where) to document our testing implementation which is using it | 15:31 |
| AJaeger | but might have been that one job failure was for the broken ovh-bhs cloud which self-healed | 15:31 |
| fungi | ahh | 15:31 |
| fungi | and yeah, sorry i wasn't on the computer before that fixed itself. i'm curious to know what went wrong in ovh | 15:32 |
| fungi | maybe they were performing network maintenance or something | 15:32 |
| AJaeger | fungi: no worries - it was soo idle that I just documented it here ;) | 15:32 |
| fungi | oh! | 15:32 |
| fungi | as in you've already written some documentation? | 15:33 |
| fungi | or are you talking about ovh? | 15:33 |
| AJaeger | fungi: I meant: Documented the ovh-bhs failure on the irc channel | 15:33 |
| fungi | okay, cool. i need to go run some errands for a couple hours, but will give the announcement and test documentation some thought and see what i come up with | 15:33 |
| fungi | and then will be around the rest of the day in case there's more we need to test/fix too | 15:33 |
| fungi | thanks for keeping an eye on it on your saturday! | 15:34 |
| AJaeger | fungi, just one question: I was thinking of shutting down ovh-bhs with lowering quote on it - but had to self-approve my own change. | 15:34 |
| AJaeger | IF there would have been real load, I would have been tempted - is that kind of self-approval in such situations fine or shouldn't I think of that? | 15:35 |
| fungi | AJaeger: in situations like this, self-approval is fine if there's nobody else around and you're just dropping capacity temporarily. especially at low-volume times | 15:35 |
| AJaeger | it was so low-volume that it wasn't needed ;) Ok, thanks - and we can always revert;) | 15:36 |
| fungi | not involving others when there's someone around to help is a bit of a problem, but if you're stuck by yourself, i'd rather people feel empowered to fix what they can and we can work out later whether it was the right call given the circumstances | 15:37 |
| AJaeger | argh, bindep repo is broken with the other-requirements file ;( | 15:37 |
| fungi | oh wow | 15:37 |
| fungi | at least that's fixable | 15:37 |
| AJaeger | yep, looking at it... | 15:37 |
| fungi | i bet it doesn't have any database packages | 15:38 |
| AJaeger | fungi, you won the bet ;) | 15:38 |
| AJaeger | http://logs.openstack.org/83/288883/1/check/gate-bindep-pypy/bb95447/console.html#_2016-03-05_15_31_26_212 | 15:38 |
| fungi | which we've basically introduced as a requirement since we're using the same job-template regardless of whether a project has database-oriented tests | 15:38 |
| fungi | so, yeah, probably any project with an other-requirements.txt file and using our typical tox job-templates is likely to need those added | 15:39 |
| fungi | i probably want to think up a solid plan for solving that soon, because when we announce bindep integration in our jobs the flood gates will open on projects adding other-requirements.txt files and i'd rather not tell them that they have to add database packages even if they don't have any database-using tests | 15:40 |
| AJaeger | fungi, I agree - so for our initial documentation, let's discourage other-requirements | 15:40 |
| *** dims has quit IRC | 15:41 | |
| fungi | maybe i'll have a grand solution in a couple hours when i get back | 15:41 |
| fungi | bbiaw | 15:41 |
| AJaeger | ;) | 15:42 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Fix other-requirements https://review.openstack.org/288904 | 15:42 |
| *** dims has joined #openstack-infra | 15:43 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/release-tools: Fix other-requirements https://review.openstack.org/288905 | 15:45 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Fix other-requirements https://review.openstack.org/288904 | 15:45 |
| dhellmann | AJaeger, fungi : where can I go to read more about bindep? I have no idea what it is. It's not showing up in a search of infra specs. | 15:46 |
| AJaeger | dhellmann: http://docs.openstack.org/infra/bindep/ | 15:47 |
| dhellmann | AJaeger : thanks | 15:47 |
| AJaeger | dhellmann: but major change here is that with going from bare-trusty to ubuntu-trusty we use a diskimage-builder to setup the images | 15:48 |
| AJaeger | Advantage is that we now can run ubuntu-trusty on all clouds while bare-trusty was rackspace only | 15:48 |
| *** _nadya_ has joined #openstack-infra | 15:48 | |
| AJaeger | We install packages now at gate setup. There's a default package list in project-config/jenkins/data/bindep-fallback.txt - and projects can use their own list called other-requirments.txt. See my changes above for bindep and release-tools.. | 15:49 |
| dhellmann | ok, I had heard of it but didn't know anything about it so was a bit alarmed to start seeing comments like "all the requirements jobs are failing" or whatever and wanted to learn a bit more so I could participate in fixing things (assuming you don't have it worked out by monday) | 15:49 |
| AJaeger | dhellmann: the fix was https://review.openstack.org/288858 | 15:50 |
| dhellmann | AJaeger : how do I make sure that a job like the release announce job, that uses release-tools but isn't triggered by a commit to that repo, has the right dependencies installed? | 15:50 |
| dhellmann | for now the release part of the tools is still being run locally, but that will have a similar requirement. merging something to openstack/releases will eventually run a job that installs release-tools and then runs some of the scripts | 15:52 |
| AJaeger | dhellmann: best discuss with fungi about this | 15:52 |
| dhellmann | AJaeger : ok, it'll wait until next week then | 15:52 |
| * AJaeger checks that job now | 15:53 | |
| AJaeger | dhellmann: Plan B would be to do something like install-distro-packages macro does and add your own package list for this case - or like install-distro-packages-fallback-test, hardcoding a list. See jenkins/jobs/macros.yaml | 15:55 |
| *** Qiming has quit IRC | 15:55 | |
| dhellmann | AJaeger : ok, building it into the job makes sense | 15:55 |
| AJaeger | dhellmann: thanks for raising this. I'm sure there are some more situations where we need to extend the current setup... | 15:56 |
| AJaeger | fungi, please read scrollback later ^ | 15:56 |
| *** ddieterly has joined #openstack-infra | 15:57 | |
| * AJaeger will be back later... | 15:57 | |
| *** thorst has joined #openstack-infra | 16:01 | |
| *** dims has quit IRC | 16:04 | |
| *** ddieterly has quit IRC | 16:05 | |
| *** thorst has quit IRC | 16:08 | |
| *** thorst has joined #openstack-infra | 16:09 | |
| *** max_lobur has quit IRC | 16:10 | |
| *** ddieterly has joined #openstack-infra | 16:11 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Allow filtering of worklists and boards by story or task https://review.openstack.org/288910 | 16:12 |
| *** dims has joined #openstack-infra | 16:16 | |
| *** thorst has quit IRC | 16:17 | |
| *** dims has quit IRC | 16:18 | |
| *** Jeffrey4l has quit IRC | 16:19 | |
| *** ddieterly is now known as ddieterly[away] | 16:23 | |
| *** pradk has quit IRC | 16:38 | |
| *** max_lobur has joined #openstack-infra | 16:43 | |
| *** asselin__ has joined #openstack-infra | 16:46 | |
| *** thorst has joined #openstack-infra | 16:52 | |
| *** armax has quit IRC | 17:00 | |
| *** armax has joined #openstack-infra | 17:01 | |
| *** ddieterly[away] is now known as ddieterly | 17:06 | |
| *** fedexo has joined #openstack-infra | 17:07 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Only create 1024MB swap in tripleo-ci https://review.openstack.org/288827 | 17:08 |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/tripleo-ci: Allow the continer job to run again https://review.openstack.org/288915 | 17:12 |
| *** sdake_ has joined #openstack-infra | 17:17 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-askbot: Add captcha to the askbot venv https://review.openstack.org/288916 | 17:17 |
| *** asselin has joined #openstack-infra | 17:17 | |
| *** sdake has quit IRC | 17:18 | |
| *** ddieterly is now known as ddieterly[away] | 17:20 | |
| *** asselin__ has quit IRC | 17:20 | |
| *** ddieterly[away] has quit IRC | 17:23 | |
| *** jamesmcarthur has joined #openstack-infra | 17:25 | |
| *** sdake has joined #openstack-infra | 17:27 | |
| *** sdake_ has quit IRC | 17:29 | |
| *** sdake_ has joined #openstack-infra | 17:31 | |
| *** sarob has joined #openstack-infra | 17:32 | |
| *** ddieterly has joined #openstack-infra | 17:34 | |
| *** sdake has quit IRC | 17:34 | |
| *** sarob has quit IRC | 17:36 | |
| *** ihrachys has quit IRC | 17:40 | |
| *** ihrachys has joined #openstack-infra | 17:41 | |
| *** ihrachys has quit IRC | 17:45 | |
| *** e0ne has joined #openstack-infra | 17:45 | |
| *** esikachev has joined #openstack-infra | 17:46 | |
| *** armax has quit IRC | 17:47 | |
| *** max_lobur has quit IRC | 17:49 | |
| *** esikachev has quit IRC | 17:50 | |
| *** salv-orlando has joined #openstack-infra | 17:52 | |
| fungi | dhellmann: AJaeger: short story is this is supposed to be transparent for projects without a other-requirements.txt file in them | 17:52 |
| fungi | the problem with the requirements repo was that someone had added that file to it | 17:53 |
| fungi | and it wasn't tested, so was broken | 17:53 |
| nibalizer | mtreinish: started up the subunit worker on subunit02 | 17:53 |
| *** jamesmcarthur has quit IRC | 17:54 | |
| fungi | dhellmann: AJaeger: so when the jobs changed to support people adding that file later (something we don't want them doing prematurely until we're sure this is a good path forward and doesn't need further tweaking), it insta-broke requirements jobs that tried to use that file | 17:54 |
| dhellmann | fungi : ah, so someone jumped the gun adding that file to the requirements repo? | 17:54 |
| fungi | dhellmann: more or less | 17:55 |
| *** salv-orl_ has quit IRC | 17:55 | |
| fungi | i think it was added for someone's local convenience, or the thought that it might be convenient for local use for some devs, but never actually tested to confirm it was correct | 17:55 |
| fungi | dhellmann: another fix would have been for us to just delete that file from the repo | 17:55 |
| *** _nadya_ has quit IRC | 17:56 | |
| fungi | the goal was for this transition not to require any changes to any existing projects, that impact was an oversight | 17:56 |
| fungi | out of the ~1k projects in gerrit, something like 4 have apparently added an other-requirements.txt file | 17:57 |
| *** mriedem has joined #openstack-infra | 17:57 | |
| fungi | and the requirements repo was one of them | 17:57 |
| dhellmann | ok, cool | 17:57 |
| dhellmann | I'll read the docs about the tool this week so I know more about it | 17:58 |
| fungi | dhellmann: making a big announcement has the potential to backfire, since the announcement would be "don't use this yet" and projects are almost certain to decide to start doing it then | 17:58 |
| dhellmann | oh, of course :-) | 17:58 |
| *** ddieterly is now known as ddieterly[away] | 17:58 | |
| fungi | we still have some rough edges around in-job database setup which we've worked around for now but will make the other-requirements.txt files unnecessarily list things which those projects don't actually need, so i want to finish addressing that piece before we declare open season | 17:59 |
| fungi | also, some deep soul searching this morning has brought me to the conclusion that "opportunistic testing" is fragile and counter to proper ci, so i think i'm going to start a -dev ml thread to discuss ways we can resolve that | 18:00 |
| fungi | with regards to things like nova unit tests going "oh, there's no database for me to use, i'll just skip these tests then" | 18:01 |
| *** jamesmcarthur has joined #openstack-infra | 18:01 | |
| fungi | i'd love to come up with a good answer for devs to be able to run whatever tests their local environments support while still making it a hard failure in the ci if the needs of a particular set of tests aren't met | 18:01 |
| clarkb | fungi: should we go ahead and bump the osic max-servers number now? I can write that patch if you want to do it | 18:02 |
| fungi | clarkb: sounds like a great time, yes | 18:02 |
| AJaeger | clarkb, fungi: Could you review the two other broken other-requirements files, please? https://review.openstack.org/#/c/288904/ for bindep | 18:02 |
| AJaeger | https://review.openstack.org/#/c/288905/ for release-tools - dhellmann, feel free to +2 as well ;) | 18:03 |
| AJaeger | clarkb: go for it... | 18:03 |
| *** harlowja_at_home has joined #openstack-infra | 18:03 | |
| AJaeger | Oh, we need to backport the requirements change to all branches ;( | 18:04 |
| AJaeger | Let me do that as well... | 18:04 |
| fungi | AJaeger: i'm starting to worry the other-requirements.txt files in this handful of repos is going to set a bad example or cause it to be blindly copied around. maybe better to just propose deletion of them for now? | 18:04 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Use entire osic quota https://review.openstack.org/288922 | 18:05 |
| fungi | then we can add them back later once we work out the database solution so that we're not claiming they all use mysql and postgres for tests when few (if any) actually do \ | 18:05 |
| AJaeger | fungi: I was wondering that as well - ok, let me propose removing them... | 18:05 |
| clarkb | AJaeger: fungi on https://review.openstack.org/#/c/288904/2 we use mysql-python and not PyMySQL? | 18:05 |
| AJaeger | clarkb: see project-config/jenkins/data/bindep* | 18:06 |
| clarkb | also I am not sure that is the right package for psycopg2 | 18:06 |
| clarkb | oh! it is because we use the client directly in the jobs | 18:06 |
| AJaeger | clarkb: then the bindep-fallback list is wrong | 18:06 |
| AJaeger | clarkb: another reason to delete the file? ;) | 18:06 |
| fungi | AJaeger: well, not wrong | 18:06 |
| *** ddieterly[away] is now known as ddieterly | 18:07 | |
| fungi | clarkb: to support the handful of projects (like nova, ironic) which expect mysql and postgres to be set up with test databases so that they don't skip some tests, the job-templates need the databases installed to be able to set them up | 18:07 |
| fungi | i doubt that libmysqlclient-dev is actually required there | 18:07 |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Remove other-requirements.txt https://review.openstack.org/288923 | 18:07 |
| fungi | macros in some of the job templates are using the command-line db clients to set up database permissions to support the projects which use them, mimicking how we preconfigure the databases on bare-trusty images | 18:08 |
| clarkb | those dev headers are used by mysql-python or is python-mysql to link against the C lib | 18:08 |
| fungi | clarkb: i think neither | 18:09 |
| fungi | clarkb: i think libmysqlclient-dev is just being copied into that list needlessly | 18:09 |
| clarkb | libpq-dev is the one that psycopg2 links against | 18:09 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Remove other-requirements.txt https://review.openstack.org/288904 | 18:09 |
| fungi | clarkb: in 288904 those entries are being added purely so that when the job macro to set database permissions runs (even though none of bindep's tests are ever going to use those databases) the job doesn't fail | 18:10 |
| clarkb | gotcha | 18:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/release-tools: Remove other-requirements.txt https://review.openstack.org/288905 | 18:11 |
| fungi | which is why i'm more in favor of just deleting stray other-requirements.txt files from projects in the short term | 18:11 |
| clarkb | +1 | 18:11 |
| clarkb | or +2 as it goes | 18:11 |
| fungi | until we can work out how to make projects not need to declare false dependencies on database services and clients purely because it was easy to reuse the same job-templates for now | 18:11 |
| clarkb | fungi: I think the macro to install and configure the dbs can vendor their own package installs | 18:12 |
| clarkb | then if individual projects need headers to use libs against them they declair those in other requirements? | 18:12 |
| clarkb | maybe? | 18:12 |
| fungi | my only remaining worry with deleting it from the requirements repo is that one of its jobs may be running on devstack-trusty and calling bindep to decide what packages to install, rather than being there purely for convenience (though if that were the case, i'd have expected the invalid entry in the file to be causing problems for that job). anyway the changes to delete the file should tell us | 18:13 |
| fungi | pretty quickly | 18:13 |
| *** ddieterly is now known as ddieterly[away] | 18:13 | |
| fungi | clarkb: my concern there is that projects may silently grow dependencies on packages used by those macros, either directly or indirectly | 18:14 |
| clarkb | yes | 18:14 |
| fungi | what i'm starting to think we ultimately need is a version of those job-templates which use databases to run their tests, and another version for the masses | 18:14 |
| clarkb | an alternative woud be to have the projects declare the deps and install first, then if mysql/postgres/whatever is present configure it | 18:14 |
| fungi | but that means figuring out which projects' tests are actually using the databases we preconfigure on bare-trusty | 18:15 |
| AJaeger | fungi, I created a tox-nodb macro already for individual tox invocations | 18:15 |
| AJaeger | and use that for manuals | 18:15 |
| AJaeger | but yes, we need to figure this out in each case ;( | 18:15 |
| fungi | AJaeger: yep, in the end we probably want that inverted so that the normal macros are the ones without extra database setup | 18:15 |
| *** keedya_ has quit IRC | 18:16 | |
| *** keedya has joined #openstack-infra | 18:16 | |
| fungi | and just the handful of projects which use preconfigured databases for some tests use the special template which does that part at the start of the job (after package installs but before revoking sudo) | 18:16 |
| AJaeger | fungi, just a global rename ;) | 18:16 |
| *** Sukhdev has joined #openstack-infra | 18:16 | |
| AJaeger | fungi, we can add a tox-db macro - or however it's called, move over and rename | 18:17 |
| *** jamesmcarthur has quit IRC | 18:17 | |
| clarkb | nibalizer: fungi AJaeger https://review.openstack.org/#/c/288922/ should be ready | 18:17 |
| clarkb | as is https://review.openstack.org/#/c/288832/ | 18:17 |
| *** thorst has quit IRC | 18:18 | |
| AJaeger | dhellmann: requirements changes are https://review.openstack.org/#/c/288924 and https://review.openstack.org/#/c/288923 - could you help pushing them through, please | 18:18 |
| fungi | AJaeger: if only it were that simple. the issue is that the "safe" position is to have everyone use the version with the databases preconfigured until we can determine which ones don't need it, and that seems like it's going to be tough to figure out in light of the "opportunism" in a lot of our database tests | 18:18 |
| * AJaeger trusts clarkb and hands out a +2 | 18:18 | |
| *** thorst has joined #openstack-infra | 18:18 | |
| AJaeger | fungi, yes I know - have a look at https://review.openstack.org/#/q/topic:bindep+status:open . | 18:19 |
| fungi | because things like nova unit tests will run perfectly fine without database setup today, but unless you actually compare test counts or look in the list of tests they run, you won't know they're missing some | 18:19 |
| AJaeger | I hope I updated so far correctly for the non-db case in these. | 18:19 |
| nibalizer | clarkb: ooo! | 18:19 |
| *** e0ne has quit IRC | 18:19 | |
| *** harlowja_at_home has quit IRC | 18:20 | |
| fungi | what i want is a solution to make tests like that _not_ opportunistic, at least when we run ci jobs | 18:20 |
| fungi | though that also means devs and the gate running tests in slightly different ways | 18:20 |
| clarkb | fungi: maybe the thing to do is put the db setup in tox itself? | 18:20 |
| clarkb | then it fails if it can't do its thing and project notices | 18:21 |
| clarkb | this requires devs to have a mysql to point nova at but that isn't too hard | 18:21 |
| fungi | clarkb: i also thought about that, but i'm not keen on the idea of opening projects up to running arbitrary shell as root in their gate jobs | 18:22 |
| fungi | i worry that will turn into the swiss-army-knife of terrible solutions for them | 18:22 |
| fungi | at least the bindep step running as root is consistently running a set of preinstalled tools against a declarative data file, not just running whatever a dev decided to tack onto the tox.ini in a special as-root env | 18:23 |
| clarkb | yup | 18:23 |
| fungi | some jobs (e.g. functional and integration testing) it's inevitable, but i fear projects getting to the point where we're making world-writeable directories because their unit tests need to write to arbitrary file locations on the system or something | 18:24 |
| fungi | too much flexibility in those jobs would be detrimental | 18:25 |
| fungi | AJaeger: i want to see if those requirements changes pass, mainly because i worry that something like the requirements integration test script might have actually been using that file | 18:26 |
| fungi | in which case this gets slightly uglier | 18:26 |
| lifeless | those would not be unit tests | 18:26 |
| fungi | lifeless: more or less my point, yeah | 18:27 |
| *** thorst has quit IRC | 18:27 | |
| lifeless | they also wouldn't be well isolated | 18:27 |
| lifeless | which is IMO a bigger issue than a name | 18:27 |
| fungi | right, the "unit" tests in nova and ironic which do opportunistic database interactions now are arguably also not really unit tests | 18:27 |
| AJaeger | So far coverage, python27, python34, pypy are setup for db access - others not. We could create two set of templates one with db, one without and let projects migrate... | 18:27 |
| *** sdake_ is now known as sdake | 18:27 | |
| fungi | AJaeger: yeah, that is what i'm assuming we're probably going to have to do | 18:28 |
| AJaeger | bindep (change 288904) and release-tools (change 288905) passed the check | 18:29 |
| *** [1]Thelo has joined #openstack-infra | 18:29 | |
| AJaeger | lifeless: I think you added other-requirements to some of the 4 repos, do you want to review the removal that I'm currently doing? https://review.openstack.org/#/q/status:open+topic:bindep lists these and a few other changes... | 18:30 |
| *** sputnik13 has joined #openstack-infra | 18:31 | |
| fungi | AJaeger: so this coming week i'll get changes up to switch the projects in question over to versions of unit test job-templates which don't perform database setup, and then we can add other-requirements.txt files back to them (the readditions will also be self-testing, which will be helpful) | 18:31 |
| *** Thelo has quit IRC | 18:31 | |
| lifeless | AJaeger: removal? why? | 18:31 |
| *** [1]Thelo is now known as Thelo | 18:31 | |
| AJaeger | lifeless: see backscroll | 18:32 |
| AJaeger | lifeless: we discussed this 40 mins or so ago | 18:32 |
| lifeless | AJaeger: sunday morning before a plane trip... unless its urgent, I'll read tomorrow | 18:32 |
| AJaeger | lifeless: read tomorrow ;) | 18:33 |
| lifeless | AJaeger: the only other-requirements I added, from memory, was pbr and requirements, and their scripts use it, so it should fail gate tests if you try to remove it | 18:33 |
| fungi | lifeless: it's non-urgent. temporary removals of files not used in existing jobs (assuming the test for them pass) which will be added back safely in a self-testing manner in a few days | 18:33 |
| AJaeger | lifeless: we can revert the removal once infra is ready - right now, they cause some problems... | 18:33 |
| openstackgerrit | Merged openstack-infra/project-config: Use entire osic quota https://review.openstack.org/288922 | 18:34 |
| *** e0ne has joined #openstack-infra | 18:34 | |
| openstackgerrit | Merged openstack-infra/bindep: Remove other-requirements.txt https://review.openstack.org/288904 | 18:34 |
| AJaeger | lifeless: safe travels! | 18:34 |
| fungi | AJaeger: lifeless: if some jobs for those repos are actually using those files (which i suspected they might) then we'll just have to correct them in-place and add a few things so that they're sufficient for also running all the python jobs too, until we can clean that up | 18:35 |
| *** armax has joined #openstack-infra | 18:35 | |
| AJaeger | fungi, indeed requirements repo uses the other-requirements file. | 18:36 |
| AJaeger | my tests are failing | 18:36 |
| lifeless | AJaeger: thanks; just to syd for the bug smash | 18:36 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: IPv4 network isolation testing for HA and Ceph https://review.openstack.org/288163 | 18:37 |
| lifeless | fungi: not sure what cleanup is needed, but yeah - today is bad. | 18:37 |
| fungi | lifeless: right, we were trying to do it today to avoid unforeseen impact while everyone was submitting patches during the week | 18:37 |
| fungi | okay, so we'll have to go back to your original plan of adding all the other packages necessary to support the job-templates instantiated on it, and maybe also a big comment around those stating that they're temporary additions, and also a comment in the file reminding people not to go copying that pattern around until we're better able to support it in all our jobs | 18:38 |
| fungi | AJaeger: ^ | 18:38 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Remove argparse from requirements https://review.openstack.org/270414 | 18:38 |
| AJaeger | fungi, we're only discussing requirements repo, correct? | 18:38 |
| fungi | AJaeger: right (or any others where the temporary removal fails because they had some custom job which used bindep to parse that file in their repo) | 18:39 |
| fungi | a future to-do was to come up with a good recommended pattern for per-job other-requirements.txt entries using specially-named tags | 18:40 |
| AJaeger | ok, next iteration for requirements ;) | 18:40 |
| clarkb | nibalizer: can haz grafana too? https://review.openstack.org/#/c/288832/ | 18:40 |
| AJaeger | per-job other-requirements is indeed another optimization - but I'm not up for the drawing board today | 18:40 |
| nibalizer | yep thats yaml | 18:41 |
| fungi | but right now the other-requirements.txt file will have to include all distro packages for any job on that project incorporating the install-distro-packages macro | 18:41 |
| clarkb | nibalizer: that is basically what I said as I replaced vexxhost with osic | 18:41 |
| *** e0ne has quit IRC | 18:43 | |
| AJaeger | fungi, the requirements repo only has an ubuntu version, should I try updating it for CentOS as well? | 18:43 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Send ansible-puppet timing information to graphite https://review.openstack.org/285740 | 18:44 |
| *** dizquierdo has quit IRC | 18:45 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Comment other-requirements.txt https://review.openstack.org/288928 | 18:46 |
| *** e0ne has joined #openstack-infra | 18:46 | |
| fungi | AJaeger: nah, i wouldn't bother adding more than necessary to that file for now | 18:48 |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Comment other-requirements.txt https://review.openstack.org/288928 | 18:48 |
| AJaeger | good ;) | 18:48 |
| fungi | just the minimum entries to get the packages and database macros to run successfully | 18:48 |
| *** hdd has quit IRC | 18:49 | |
| lifeless | fungi: put the temporary requirements in a dedicated profile ? | 18:51 |
| *** ddieterly[away] is now known as ddieterly | 18:51 | |
| lifeless | fungi: that is what they are there for.... | 18:51 |
| AJaeger | fungi, clarkb, dhellmann: https://review.openstack.org/288928 is the backport of requirements change to stable/liberty so that it works in our gates. Please review and approve | 18:52 |
| *** armax has quit IRC | 18:52 | |
| openstackgerrit | Merged openstack-infra/project-config: Grafana for OSIC https://review.openstack.org/288832 | 18:52 |
| *** ddieterly has quit IRC | 18:52 | |
| clarkb | https://jenkins01.openstack.org/job/gate-tempest-dsvm-cells/3752/console https://jenkins05.openstack.org/job/gate-tempest-dsvm-full/30302/console https://jenkins07.openstack.org/job/gate-requirements-integration-dsvm-resolver/367/console | 18:52 |
| AJaeger | clarkb: woot! | 18:53 |
| clarkb | https://jenkins07.openstack.org/job/gate-tempest-dsvm-full-devstack-plugin-ceph-nv/1880/console is another | 18:55 |
| *** sputnik13 has quit IRC | 18:55 | |
| *** e0ne has quit IRC | 18:57 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Collect status of all nested stacks in resource-list https://review.openstack.org/286062 | 18:59 |
| *** e0ne has joined #openstack-infra | 19:01 | |
| fungi | lifeless: i eventually want the install-distro-packages job macro to support job-specific bindep profiles, but the initial simple design relies on the list to contain the set of all packages needed by all jobs which use that macro | 19:01 |
| fungi | in the default profile | 19:02 |
| clarkb | https://jenkins07.openstack.org/job/gate-tempest-dsvm-full-devstack-plugin-ceph-nv/1880/console | 19:02 |
| clarkb | er | 19:02 |
| clarkb | http://grafana.openstack.org/dashboard/db/nodepool-osic | 19:03 |
| fungi | AJaeger: 288928 looks like master to me | 19:04 |
| AJaeger | fungi: it's failing at https://jenkins01.openstack.org/job/gate-requirements-python27/2939/console - and I'm confused | 19:05 |
| fungi | looking | 19:05 |
| AJaeger | yes, something looks strange there ;( | 19:06 |
| *** sdake_ has joined #openstack-infra | 19:06 | |
| AJaeger | looking at https://review.openstack.org/#/c/288929/1/other-requirements.txt it looks fine - but looking at a version using git review -d ...;git show it looks broken ;( | 19:07 |
| AJaeger | am I mixing up numbers? | 19:08 |
| * AJaeger double checks | 19:08 | |
| *** sdake has quit IRC | 19:08 | |
| AJaeger | 288928 is master 288929 is stable/liberty | 19:09 |
| *** sdake has joined #openstack-infra | 19:10 | |
| fungi | AJaeger: the 288929 commit looks sane to me. i wonder if there was already something broken with stable/liberty requirements unit tests? | 19:11 |
| AJaeger | fungi, I leave this to the requirements experts | 19:12 |
| fungi | yeah, something is causing it to fail several of its negative parser checks | 19:12 |
| *** sdake_ has quit IRC | 19:12 | |
| AJaeger | lifeless, dhellmann : could you check 288929 tomorrow, please? I don't understand why it suddenly fails - and we need this backport to unbreak stable/liberty requirements | 19:12 |
| fungi | pkg_resources._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "':///foo#'" | 19:13 |
| fungi | except, that seems like what it _should_ be doing for the openstack_requirements.tests.test_requirement.TestToReqs.test_not_urls test | 19:13 |
| fungi | so why is that a test failure? | 19:14 |
| AJaeger | fungi, no idea - and I'm signing off now... | 19:15 |
| AJaeger | fungi, have a great saturday! | 19:16 |
| fungi | thanks for working on this AJaeger and have a good evening/weekend! | 19:16 |
| AJaeger | thanks, fungi | 19:16 |
| *** e0ne has quit IRC | 19:18 | |
| *** thorst has joined #openstack-infra | 19:18 | |
| fungi | looks like it's been a couple weeks since that job ran on stable/liberty so it's possible something has broken how the exceptions are being interpreted in those specific tests | 19:20 |
| *** ddieterly has joined #openstack-infra | 19:21 | |
| fungi | i'm going back through the git history on requirements master now to see if there was a fix merged for something like this in the past couple weeks that we might need to backport and squash into | 19:22 |
| *** e0ne has joined #openstack-infra | 19:22 | |
| fungi | taking a while. feature freeze seems to have been a popular time to get requirements changes crammed in | 19:24 |
| fungi | aha, https://review.openstack.org/285838 "Tolerate both new and old setuptools" | 19:25 |
| fungi | i'll get that backported to stable/liberty and squashed together with the other-requirements.txt update | 19:25 |
| *** ihrachys has joined #openstack-infra | 19:26 | |
| *** e0ne has quit IRC | 19:28 | |
| clarkb | fungi: ok I sent mail to osic letting them know we are live with links to a successful tempest job run and the grafana dashboard | 19:29 |
| *** thorst has quit IRC | 19:29 | |
| clarkb | tempest full took 37 minutes so that looks good | 19:30 |
| *** ihrachys has quit IRC | 19:30 | |
| *** e0ne has joined #openstack-infra | 19:32 | |
| fungi | dhellmann: mriedem: so stable/liberty requirements unit tests were broken for a week or so already. latest patchset of https://review.openstack.org/288929 combines a backport of AJaeger's fix from today for our recent job changes with a backport of dims's patch from earlier for dealing with new setuptools | 19:33 |
| *** achanda has joined #openstack-infra | 19:34 | |
| *** achanda has quit IRC | 19:34 | |
| fungi | clarkb: excellent news--thanks!!! | 19:34 |
| *** e0ne has quit IRC | 19:35 | |
| *** e0ne has joined #openstack-infra | 19:38 | |
| clarkb | oh I forgot to mention the thing about specifying the cinder volume attach point | 19:40 |
| mriedem | fungi: looking | 19:40 |
| *** e0ne has quit IRC | 19:41 | |
| mriedem | fungi: AJaeger: question inline | 19:43 |
| *** e0ne has joined #openstack-infra | 19:46 | |
| clarkb | ok updated email sent with volume attach details | 19:46 |
| *** e0ne has quit IRC | 19:48 | |
| *** Sukhdev has quit IRC | 19:50 | |
| *** Sukhdev has joined #openstack-infra | 19:50 | |
| *** e0ne has joined #openstack-infra | 19:52 | |
| *** claudiub|2 has joined #openstack-infra | 19:53 | |
| *** claudiub has quit IRC | 19:56 | |
| *** e0ne has quit IRC | 19:58 | |
| *** zul has quit IRC | 20:01 | |
| *** e0ne has joined #openstack-infra | 20:03 | |
| *** e0ne has quit IRC | 20:06 | |
| *** _nadya_ has joined #openstack-infra | 20:09 | |
| *** e0ne has joined #openstack-infra | 20:11 | |
| *** ddieterly is now known as ddieterly[away] | 20:11 | |
| dhellmann | AJaeger , fungi : thanks for digging into that. I'll wait for the answer to mriedem's question before approving | 20:14 |
| *** zul has joined #openstack-infra | 20:15 | |
| fungi | checking | 20:17 |
| fungi | mriedem: i think AJaeger's previous patchset was actually squashing two changes from master already but didn't indicate that in the commit message | 20:18 |
| fungi | he's gone for the day but i'll see if i can fix up the commit message to better reflect that | 20:19 |
| AJaeger | mriedem: see https://review.openstack.org/288928, that's the version taht is backported | 20:20 |
| AJaeger | fungi, just came back to clarify this ;) | 20:20 |
| AJaeger | fungi, yes, indeed my commit message was not clear - if you want to update this, highly appreciated. | 20:21 |
| fungi | i will do so--thanks AJaeger and enjoy your weekend! | 20:22 |
| AJaeger | thanks a lot, fungi! | 20:22 |
| *** ihrachys has joined #openstack-infra | 20:22 | |
| *** eharney has quit IRC | 20:23 | |
| *** max_lobur has joined #openstack-infra | 20:25 | |
| *** e0ne has quit IRC | 20:27 | |
| *** ihrachys has quit IRC | 20:28 | |
| *** _nadya_ has quit IRC | 20:30 | |
| *** e0ne has joined #openstack-infra | 20:30 | |
| *** e0ne has quit IRC | 20:33 | |
| *** e0ne has joined #openstack-infra | 20:37 | |
| *** thorst has joined #openstack-infra | 20:38 | |
| *** keedya has quit IRC | 20:38 | |
| *** claudiub|2 has quit IRC | 20:39 | |
| *** e0ne has quit IRC | 20:40 | |
| openstackgerrit | Merged openstack/requirements: Comment other-requirements.txt https://review.openstack.org/288928 | 20:43 |
| *** thorst has quit IRC | 20:43 | |
| *** Sukhdev has quit IRC | 20:54 | |
| fungi | mriedem: dhellmann: i've answered the questions on https://review.openstack.org/288929 and updated the commit message accordingly | 20:58 |
| *** e0ne has joined #openstack-infra | 20:58 | |
| fungi | likely not urgent as nobody'd noticed for a while that branch has been broken by newer setuptools, and there are no open patches aside from that one anyway | 21:01 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add support for ipmitool driver in Ironic https://review.openstack.org/284036 | 21:10 |
| *** ddieterly[away] has quit IRC | 21:11 | |
| *** e0ne has quit IRC | 21:11 | |
| *** fawadkhaliq has joined #openstack-infra | 21:12 | |
| *** jtomasek has quit IRC | 21:20 | |
| *** dims has joined #openstack-infra | 21:20 | |
| *** jtomasek has joined #openstack-infra | 21:23 | |
| *** Sukhdev has joined #openstack-infra | 21:23 | |
| *** Daisy has joined #openstack-infra | 21:23 | |
| *** angdraug has joined #openstack-infra | 21:24 | |
| *** |-paul-| has joined #openstack-infra | 21:24 | |
| *** Daisy has quit IRC | 21:28 | |
| *** dabukalam_ is now known as dabukalam | 21:30 | |
| *** ihrachys has joined #openstack-infra | 21:40 | |
| *** jtomasek has quit IRC | 21:41 | |
| *** thorst has joined #openstack-infra | 21:41 | |
| *** ihrachys has quit IRC | 21:44 | |
| *** thorst has quit IRC | 21:48 | |
| *** thorst has joined #openstack-infra | 21:48 | |
| *** salv-orlando has quit IRC | 21:52 | |
| *** salv-orlando has joined #openstack-infra | 21:52 | |
| *** Sukhdev has quit IRC | 21:53 | |
| *** HeOS has joined #openstack-infra | 22:04 | |
| *** ddieterly has joined #openstack-infra | 22:22 | |
| *** Jeffrey4l has joined #openstack-infra | 22:25 | |
| *** dkehn has quit IRC | 22:25 | |
| *** fawadkhaliq has quit IRC | 22:27 | |
| *** skraynev has quit IRC | 22:28 | |
| *** kambiz has quit IRC | 22:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/bindep: Do not merge! https://review.openstack.org/288943 | 22:31 |
| *** skraynev has joined #openstack-infra | 22:31 | |
| *** trey has quit IRC | 22:32 | |
| *** Clint has quit IRC | 22:32 | |
| *** dkehn has joined #openstack-infra | 22:33 | |
| *** kuromagi has quit IRC | 22:33 | |
| *** snikitin has quit IRC | 22:33 | |
| *** cburgess has quit IRC | 22:34 | |
| *** waynr has quit IRC | 22:34 | |
| *** pleia2 has quit IRC | 22:35 | |
| *** kambiz has joined #openstack-infra | 22:36 | |
| *** snikitin has joined #openstack-infra | 22:36 | |
| *** pleia2 has joined #openstack-infra | 22:37 | |
| *** kuromagi has joined #openstack-infra | 22:38 | |
| *** sdake has quit IRC | 22:38 | |
| *** Clint has joined #openstack-infra | 22:39 | |
| *** Clint has quit IRC | 22:39 | |
| *** Clint has joined #openstack-infra | 22:39 | |
| *** salv-orlando has quit IRC | 22:39 | |
| *** cburgess has joined #openstack-infra | 22:40 | |
| *** waynr has joined #openstack-infra | 22:40 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a quote button to comments https://review.openstack.org/288944 | 22:40 |
| *** trey has joined #openstack-infra | 22:45 | |
| *** ddieterly is now known as ddieterly[away] | 22:45 | |
| *** snikitin has quit IRC | 22:45 | |
| *** thorst has quit IRC | 22:46 | |
| *** thorst has joined #openstack-infra | 22:47 | |
| *** Jeffrey4l has quit IRC | 22:48 | |
| *** |-paul-| has quit IRC | 22:49 | |
| *** ddieterly[away] is now known as ddieterly | 22:50 | |
| *** snikitin has joined #openstack-infra | 22:50 | |
| *** ddieterly is now known as ddieterly[away] | 22:52 | |
| *** thorst has quit IRC | 22:55 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Allow filtering of worklists and boards by story or task https://review.openstack.org/288910 | 22:55 |
| *** nijaba_ has quit IRC | 22:57 | |
| *** amotoki has quit IRC | 22:58 | |
| *** nijaba has joined #openstack-infra | 23:03 | |
| *** nijaba has quit IRC | 23:03 | |
| *** nijaba has joined #openstack-infra | 23:03 | |
| *** sdake has joined #openstack-infra | 23:13 | |
| *** dims has quit IRC | 23:16 | |
| *** jamesmcarthur has joined #openstack-infra | 23:17 | |
| *** sdake has quit IRC | 23:21 | |
| *** ihrachys has joined #openstack-infra | 23:28 | |
| *** ihrachys has quit IRC | 23:32 | |
| *** fedexo has quit IRC | 23:35 | |
| *** salv-orlando has joined #openstack-infra | 23:40 | |
| *** baoli has joined #openstack-infra | 23:44 | |
| *** aspiers has quit IRC | 23:46 | |
| *** salv-orlando has quit IRC | 23:47 | |
| *** fawadkhaliq has joined #openstack-infra | 23:48 | |
| *** ddieterly[away] is now known as ddieterly | 23:49 | |
| *** baoli has quit IRC | 23:52 | |
| *** thorst has joined #openstack-infra | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!