| *** nadya is now known as Guest93515 | 00:00 | |
| *** dimtruck is now known as zz_dimtruck | 00:00 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Added visual indicator for add task form https://review.openstack.org/122259 | 00:01 |
|---|---|---|
| *** doug-fish has left #openstack-infra | 00:02 | |
| *** r-daneel_ has quit IRC | 00:03 | |
| *** ianw has quit IRC | 00:04 | |
| *** amcrn has joined #openstack-infra | 00:04 | |
| *** Guest93515 has quit IRC | 00:04 | |
| *** ianw has joined #openstack-infra | 00:05 | |
| *** zz_dimtruck is now known as dimtruck | 00:10 | |
| jeblair | clarkb: lgtm, and thanks for fixing the NODEPOOL_ vars bit too (i noticed that when i was reading the code) | 00:12 |
| *** xchu has joined #openstack-infra | 00:12 | |
| jeblair | clarkb: i think i just got my test environment working; i'll check out your code and try it | 00:13 |
| fungi | i assume we'll need a nodepoold restart before we're ready to try again | 00:13 |
| *** signed8bit has joined #openstack-infra | 00:13 | |
| jeblair | i think i should continue testing this locally | 00:13 |
| *** penguinRaider has quit IRC | 00:13 | |
| fungi | at least in production | 00:13 |
| fungi | i agree | 00:13 |
| jeblair | because a bug like that sort of suggests that hasn't happened yet. :( | 00:13 |
| clarkb | jeblair: ++ to more testing | 00:13 |
| clarkb | and ya it really helped me wrap my head around what was happening to refactor the code a little | 00:13 |
| fungi | i'm surprised this didn't crop up when the feature was implemented before, but it mmakes sense that the development workflow very we;; may have involved starting nodepoold fresh after each change | 00:14 |
| openstackgerrit | A change was merged to openstack-infra/config: Switch sphinxcontrib-docbookrestapi to python-jobs https://review.openstack.org/118860 | 00:14 |
| openstackgerrit | A change was merged to openstack-infra/config: Add non-voting gate-git-review-python34 job https://review.openstack.org/119964 | 00:14 |
| clarkb | fungi: ya Ithink that would've let it go through | 00:14 |
| clarkb | fungi: then it would put images in PWD whatever that is (/ because of daemonization?) | 00:14 |
| fungi | after each config change i mean | 00:14 |
| clarkb | and find them there | 00:14 |
| fungi | yep | 00:15 |
| clarkb | so it would mostly work | 00:15 |
| fungi | testing adding it to the configuration of a running nodepoold presumably never happened | 00:15 |
| *** melwitt has quit IRC | 00:16 | |
| *** signed8bit has quit IRC | 00:17 | |
| *** bdpayne has quit IRC | 00:18 | |
| jeblair | Exception: Invalid image config. Must specify either a setup script, or a diskimage to use. | 00:19 |
| jeblair | clarkb: ^ | 00:19 |
| jeblair | i think we're missing something from the config | 00:19 |
| *** achanda has quit IRC | 00:19 | |
| jeblair | clarkb: oh nevermind | 00:19 |
| jeblair | _I'm_ missing something :) | 00:19 |
| clarkb | ok :) | 00:20 |
| jeblair | it's at the end which is not where i expected it | 00:20 |
| openstackgerrit | A change was merged to openstack-infra/infra-specs: Remove templates from toctrees https://review.openstack.org/120557 | 00:20 |
| jeblair | 2014-09-23 00:20:32,635 INFO nodepool.DiskImageBuilderThread: Running disk-image-create -x --no-tmpfs -o /tmp/nodepool_dib/devstack-trusty-dib-1411431632.qcow2 ubuntu vm openstack-repos puppet nodepool-base node-devstack | 00:20 |
| jeblair | that's promising | 00:20 |
| openstackgerrit | A change was merged to openstack-infra/infra-specs: Use the current date for the copyright statement https://review.openstack.org/120558 | 00:21 |
| jeblair | so this gives no ongoing feedback until the process completes :( | 00:22 |
| jesusaurus | alifeless: im seeing behaviour similar to bug 1277538 in nodepool tests and i see you're working on http://bugs.python.org/issue19746 can you give me a quick explanation of where the api compatibility is breaking? and do you happen to know what the latest good versions of testtools/subunit are? | 00:23 |
| uvirtbot | Launchpad bug 1277538 in tempest "Non-zero exit code (2) from test listing" [Undecided,In progress] https://launchpad.net/bugs/1277538 | 00:23 |
| openstackgerrit | A change was merged to openstack-infra/infra-manual: Stop using intersphinx https://review.openstack.org/121322 | 00:23 |
| jeblair | clarkb, fungi: i aprvd clarkb's nodepool changes since it looks like they work well enough to at least start dib | 00:24 |
| fungi | awesome | 00:24 |
| lifeless | jesusaurus: what api compatibility break? | 00:24 |
| jeblair | clarkb, fungi: i have no idea how long my local test will take; so if you wanted to restart nodepool and see how far it gets, i'd be okay with that. if you want to wait to see what my results are, that's cool too, though i'm about to afk for dinner so that may be tomorrow. | 00:25 |
| jesusaurus | lifeless: is it not an api compatibility that causes the testr list to break? maybe i was misreading something | 00:25 |
| *** bhuvan_ has joined #openstack-infra | 00:25 | |
| *** bhuvan_ has quit IRC | 00:26 | |
| *** bhuvan_ has joined #openstack-infra | 00:26 | |
| clarkb | jeblair: at this point tomorrow works best for me | 00:26 |
| fungi | i'm mostly afk for the evening already, and as usual feel like i accomplished little of what i'd planned to do today. tomorrow i'll hopefully be in a better spot to help test it out | 00:26 |
| fungi | in the meantime, i'm in need of beer and cartoons instead | 00:27 |
| lifeless | jesusaurus: perhaps if you start from the beginning | 00:27 |
| *** yamahata has quit IRC | 00:27 | |
| lifeless | jesusaurus: what are you doing, what are you observing | 00:27 |
| *** gargola has quit IRC | 00:28 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Handle config updates in dib builder thread https://review.openstack.org/123283 | 00:28 |
| *** bhuvan has quit IRC | 00:28 | |
| jeblair | clarkb, fungi: dib seems to use ~/.cache to hold lots of data which may not be exactly what we want. | 00:29 |
| *** gokrokve has joined #openstack-infra | 00:29 | |
| fungi | configurable, presumably? | 00:29 |
| jesusaurus | lifeless: I am seeing behaviour similar to 1277538, I'm getting "Non-zero exit code (2) from test listing" and downgrading versions of testtools and subunit resolve the issue | 00:29 |
| lifeless | not currently | 00:29 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Move the ImageUpdater base class for clarity. https://review.openstack.org/123284 | 00:29 |
| fungi | :/ | 00:29 |
| lifeless | jesusaurus: there should be more output than just that one line | 00:30 |
| *** bhuvan_ has quit IRC | 00:30 | |
| jeblair | that's an unfortunate collision between how things should work, and how things actually do work on clouds (uselessly small / partitions) | 00:30 |
| jesusaurus | lifeless: http://logs.openstack.org/43/74943/6/check/gate-nodepool-python27/37c4157/console.html#_2014-09-22_19_22_05_451 | 00:30 |
| lifeless | fungi: a symlink in the right place will configure it | 00:30 |
| fungi | fair point | 00:31 |
| lifeless | jesusaurus: 'nodepool.tests.test_nodepool' | 00:31 |
| lifeless | jesusaurus: that failed to import | 00:31 |
| lifeless | jesusaurus: possibly a missing dependency, bad syntax, or other issue with the test module. | 00:31 |
| jesusaurus | lifeless: i was following a rabbit hole through launchpad tickets and convinced myself that issue19746 was triggering behaviour in testrepository | 00:32 |
| jeblair | fungi, clarkb: 'ps' indicates that it is actually proceeding and currently cloning git repos | 00:32 |
| jesusaurus | lifeless: i can reproduce locally, and can resolve locally by downgrading the versions as described in https://bugs.launchpad.net/tempest/+bug/1277538/comments/12 | 00:32 |
| uvirtbot | Launchpad bug 1277538 in tempest "Non-zero exit code (2) from test listing" [Undecided,In progress] | 00:32 |
| fungi | i guess some manner of logging or introspection (status dump?) would be a helpful addition to dib | 00:33 |
| clarkb | jeblair: that is great news. so better logging is something else we sould add | 00:33 |
| lifeless | jesusaurus: no, this has never been caused by testr. Testr has failed to *show things nicely* in the past, but thats a cascade error, not a causative error | 00:33 |
| jesusaurus | ah okay | 00:33 |
| clarkb | fungi: jeblair: dib has logging via set -x and it summarrizes each step | 00:33 |
| lifeless | jesusaurus: that suggests that there is something in testtools that moved or whatever | 00:33 |
| clarkb | I think we just need to incorporate that into nodepool somehow or have the dib command write where we want it directly | 00:33 |
| lifeless | jesusaurus: just do this | 00:33 |
| openstackgerrit | A change was merged to openstack-infra/config: Shoebox: Binary data archiving library. https://review.openstack.org/120864 | 00:34 |
| lifeless | jesusaurus: python -c 'import nodepool.tests.test_nodepool' | 00:34 |
| lifeless | jesusaurus: and see whwat happens | 00:34 |
| *** oomichi has joined #openstack-infra | 00:34 | |
| fungi | clarkb: oh, if it's shell, that's fine then. maybe filtering it through something to add timestamps would be useful though | 00:34 |
| jesusaurus | lifeless: whee "ImportError: No module named MySQLdb" | 00:35 |
| openstackgerrit | A change was merged to openstack-infra/config: Add new project monasca-statsd https://review.openstack.org/119403 | 00:35 |
| jesusaurus | not sure what isnt being imported in the that test output though | 00:35 |
| lifeless | jesusaurus: and there is your issue | 00:35 |
| lifeless | jesusaurus: http://bugs.python.org/issue19746 is about getting that error exposed to you | 00:36 |
| jeblair | clarkb, fungi: yeah, right now it's using the subprocess.communicate method, which returns everything as one blob; we just need to switch that to line-by-line | 00:36 |
| *** dannywilson has quit IRC | 00:36 | |
| *** HenryG has quit IRC | 00:36 | |
| *** dannywilson has joined #openstack-infra | 00:37 | |
| jesusaurus | lifeless: okay thats my local issue. is there a way to figure out what my issue is in that console log? im assuming the test slave has its needed modules installed | 00:37 |
| fungi | right, we'd presumably need a stream reader and poll the fd from it or something along those lines, at which point we can just stuff those through the logger | 00:37 |
| clarkb | jesusaurus: did you run the python -c command in your virtualenv? | 00:37 |
| clarkb | jesusaurus: if you didn't then that module may not be found because it isn't installed globally | 00:37 |
| jesusaurus | clarkb: yeah | 00:37 |
| jesusaurus | clarkb: i decided against installing mysql locally to run the tests, so i just pushed it up, and was trying to debug that cryptic error in the console log | 00:38 |
| clarkb | jeblair: fungi you guys ok with leaving nodepool as is for the evening? we will just try to exec dib and fail a lot | 00:40 |
| clarkb | shouldn't be too terrible | 00:41 |
| *** dannywilson has quit IRC | 00:41 | |
| lifeless | jesusaurus: you don't need mysql installed to load the tests, but you do need the module installed | 00:41 |
| fungi | i'll look at it again first thing in the morning just in case | 00:41 |
| *** MarkAtwood has joined #openstack-infra | 00:43 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add xstatic-angular-fileupload support https://review.openstack.org/120101 | 00:44 |
| *** marcoemorais has quit IRC | 00:44 | |
| openstackgerrit | A change was merged to openstack-infra/config: Pin jenkins jobs of groups portal to precise https://review.openstack.org/123211 | 00:46 |
| *** salv-orlando has quit IRC | 00:46 | |
| boris-42 | clarkb hey there | 00:46 |
| boris-42 | clarkb could you please take a look at small bug fix for rally job this https://review.openstack.org/#/c/120815/2 | 00:46 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Switch gerrit to project-config https://review.openstack.org/122894 | 00:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Doc updates for project-config https://review.openstack.org/122908 | 00:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Switch specs to project-config https://review.openstack.org/122904 | 00:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Switch gerritbot to project-config https://review.openstack.org/122905 | 00:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Switch nodepool to project-config https://review.openstack.org/122902 | 00:49 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Switch accessbot to project-config https://review.openstack.org/122903 | 00:49 |
| *** david-lyle has quit IRC | 00:50 | |
| *** lcheng_ has quit IRC | 00:50 | |
| *** armax has joined #openstack-infra | 00:56 | |
| jeblair | i think the updated time in gertty is suffering from timezone confusion | 00:58 |
| jeblair | rather, the new column that displays it | 00:58 |
| *** dannywilson has joined #openstack-infra | 00:59 | |
| jeblair | i think it does the check for whether it's "today" by comparing utc to local time | 00:59 |
| *** rmcall has quit IRC | 00:59 | |
| jeblair | so i see changes that were updated on "Sep 23" because it's not "today" which is weird. | 00:59 |
| jeblair | (this is not a problem for fungi) | 00:59 |
| *** rmcall has joined #openstack-infra | 01:00 | |
| *** shashankhegde has quit IRC | 01:00 | |
| anteaya | devananda: if the program & project is listed in this file the contributor is an ATC: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml if other stipulations are also met | 01:00 |
| anteaya | annegentle: the way I was told was for opw | 01:01 |
| anteaya | reed: who was your contact from hp last time? if you email whoever sent you money last time (I don't know who that is) that person is probably your fastest route this time too | 01:02 |
| *** ChuckC_ has quit IRC | 01:02 | |
| reed | anteaya, I have no idea, I need to coordinate with annegentle | 01:02 |
| anteaya | reed: well how did you get money from hp last time? | 01:02 |
| *** rmcall has quit IRC | 01:03 | |
| anteaya | someone must have put their name on it | 01:03 |
| *** derekh has joined #openstack-infra | 01:03 | |
| reed | i don't do money :) | 01:03 |
| *** Longgeek has joined #openstack-infra | 01:04 | |
| reed | i'll sort it out | 01:04 |
| anteaya | well we are both in the same boat then, I don't do money either | 01:04 |
| pleia2 | ok, let us know if you need contacts again and we'll see what we can do | 01:04 |
| *** jyuso has joined #openstack-infra | 01:05 | |
| *** otherwiseguy has quit IRC | 01:07 | |
| *** dannywilson has quit IRC | 01:09 | |
| *** dannywilson has joined #openstack-infra | 01:10 | |
| *** dannywilson has quit IRC | 01:11 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Python based daemon management https://review.openstack.org/122890 | 01:11 |
| *** dannywilson has joined #openstack-infra | 01:11 | |
| *** Longgeek has quit IRC | 01:11 | |
| *** Longgeek has joined #openstack-infra | 01:11 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Python based daemon management https://review.openstack.org/122890 | 01:12 |
| *** otherwiseguy has joined #openstack-infra | 01:13 | |
| *** hdd has quit IRC | 01:14 | |
| *** oomichi has quit IRC | 01:16 | |
| openstackgerrit | A change was merged to openstack/requirements: Ensure requirements-py2 is synced https://review.openstack.org/122953 | 01:17 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Disable cinder with ironic https://review.openstack.org/115803 | 01:17 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Python based daemon management https://review.openstack.org/122890 | 01:17 |
| openstackgerrit | A change was merged to openstack-infra/config: Add publishing job for oslosphinx documentation https://review.openstack.org/122455 | 01:18 |
| *** ChuckC_ has joined #openstack-infra | 01:19 | |
| *** zns has joined #openstack-infra | 01:21 | |
| *** gokrokve_ has joined #openstack-infra | 01:22 | |
| *** gyee has quit IRC | 01:25 | |
| *** unicell has joined #openstack-infra | 01:25 | |
| *** gokrokve has quit IRC | 01:25 | |
| *** yaguang has joined #openstack-infra | 01:26 | |
| *** gokrokve_ has quit IRC | 01:27 | |
| *** dane_leblanc has joined #openstack-infra | 01:27 | |
| adam_g | anyone got a sec? https://review.openstack.org/#/c/123265/ 1 liner config patch, adds a missing DSG flag that was overlooked in patch earlier today | 01:27 |
| fungi | jeblair: heh, yep, i never thought twice about gertty showing me utc dates/times. timezoneless living is not without it's benefits | 01:27 |
| *** dimtruck is now known as zz_dimtruck | 01:30 | |
| *** Daisy has joined #openstack-infra | 01:32 | |
| adam_g | fungi, thanks! | 01:33 |
| *** MarkAtwood has quit IRC | 01:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add tripleo-ansible to stackforge https://review.openstack.org/121776 | 01:35 |
| *** derekh has quit IRC | 01:38 | |
| anteaya | 457 in merge-check wooo | 01:39 |
| *** nelsnelson has quit IRC | 01:39 | |
| *** arosen1 has quit IRC | 01:42 | |
| *** arosen1 has joined #openstack-infra | 01:43 | |
| *** mfer has joined #openstack-infra | 01:46 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack-dev/hacking: Updated from global requirements https://review.openstack.org/123311 | 01:54 |
| *** dane_leblanc has quit IRC | 01:54 | |
| *** dane_leblanc has joined #openstack-infra | 01:55 | |
| *** oanufriev has joined #openstack-infra | 01:56 | |
| *** julim has quit IRC | 01:58 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/os-loganalyze: Add ability to download log bundles https://review.openstack.org/122615 | 01:58 |
| *** praneshp has quit IRC | 02:01 | |
| *** zz_dimtruck is now known as dimtruck | 02:06 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable Neutron for ironic-parallel https://review.openstack.org/123265 | 02:08 |
| *** homeless has quit IRC | 02:08 | |
| *** dane_leblanc has quit IRC | 02:09 | |
| *** rmcall has joined #openstack-infra | 02:12 | |
| *** Daisy has quit IRC | 02:12 | |
| *** dims_ has quit IRC | 02:15 | |
| *** dims has joined #openstack-infra | 02:16 | |
| *** amcrn has quit IRC | 02:16 | |
| *** bdpayne has joined #openstack-infra | 02:17 | |
| *** radez_g0n3 is now known as radez | 02:19 | |
| *** dims has quit IRC | 02:20 | |
| *** signed8bit has joined #openstack-infra | 02:22 | |
| *** andreaf has quit IRC | 02:23 | |
| *** andreaf has joined #openstack-infra | 02:24 | |
| openstackgerrit | Steve Martinelli proposed a change to openstack-infra/config: Add functional test gate job for openstackclient https://review.openstack.org/122599 | 02:26 |
| openstackgerrit | Steve Martinelli proposed a change to openstack-infra/config: Enable functional testing for openstackclient https://review.openstack.org/122765 | 02:29 |
| openstackgerrit | Steve Martinelli proposed a change to openstack-infra/config: Enable voting for openstackclient functional tests https://review.openstack.org/122766 | 02:29 |
| *** mfer has quit IRC | 02:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable debug logging for tripleo ha job https://review.openstack.org/119218 | 02:37 |
| *** dannywilson has quit IRC | 02:41 | |
| *** dannywilson has joined #openstack-infra | 02:42 | |
| *** Ryan_Lane has quit IRC | 02:45 | |
| *** signed8bit has quit IRC | 02:45 | |
| *** bhuvan has joined #openstack-infra | 02:46 | |
| *** dannywilson has quit IRC | 02:46 | |
| *** HenryG has joined #openstack-infra | 02:48 | |
| *** koolhead17 has joined #openstack-infra | 02:49 | |
| *** praneshp has joined #openstack-infra | 02:49 | |
| *** praneshp_ has joined #openstack-infra | 02:50 | |
| *** praneshp_ has quit IRC | 02:50 | |
| *** praneshp_ has joined #openstack-infra | 02:50 | |
| *** dimtruck is now known as zz_dimtruck | 02:51 | |
| *** praneshp has quit IRC | 02:54 | |
| *** praneshp_ is now known as praneshp | 02:54 | |
| *** markmcclain has quit IRC | 02:54 | |
| *** marun is now known as marun_afk | 02:55 | |
| *** zz_dimtruck is now known as dimtruck | 02:56 | |
| *** KanagarajM has joined #openstack-infra | 02:58 | |
| *** achanda has joined #openstack-infra | 02:59 | |
| *** jyuso has quit IRC | 02:59 | |
| *** armax has quit IRC | 03:00 | |
| *** KanagarajM has quit IRC | 03:06 | |
| *** signed8bit has joined #openstack-infra | 03:09 | |
| *** nadya_ has joined #openstack-infra | 03:14 | |
| *** mmedvede has quit IRC | 03:14 | |
| *** marun_afk has quit IRC | 03:16 | |
| *** shashankhegde has joined #openstack-infra | 03:22 | |
| *** pcrews has quit IRC | 03:22 | |
| *** pcrews has joined #openstack-infra | 03:25 | |
| *** jyuso has joined #openstack-infra | 03:31 | |
| *** patrickeast has quit IRC | 03:32 | |
| *** pcrews has quit IRC | 03:41 | |
| *** zns has quit IRC | 03:46 | |
| *** zns has joined #openstack-infra | 03:47 | |
| *** morgan_remote_ has joined #openstack-infra | 03:50 | |
| *** marcoemorais has joined #openstack-infra | 03:50 | |
| *** marcoemorais1 has joined #openstack-infra | 03:52 | |
| *** lcheng has joined #openstack-infra | 03:54 | |
| *** marcoemorais has quit IRC | 03:55 | |
| *** arosen1 has quit IRC | 03:58 | |
| *** vdreamarkitex has quit IRC | 03:58 | |
| *** pcrews has joined #openstack-infra | 03:59 | |
| *** MarkAtwood has joined #openstack-infra | 04:01 | |
| *** unicell has quit IRC | 04:03 | |
| *** yamahata has joined #openstack-infra | 04:06 | |
| *** markmcclain has joined #openstack-infra | 04:07 | |
| *** garyh has quit IRC | 04:09 | |
| *** vinbs has joined #openstack-infra | 04:10 | |
| *** Ryan_Lane has joined #openstack-infra | 04:11 | |
| *** tnovacik has joined #openstack-infra | 04:11 | |
| *** hdd has joined #openstack-infra | 04:13 | |
| *** vinbs_ has joined #openstack-infra | 04:14 | |
| *** vinbs has quit IRC | 04:17 | |
| *** markmcclain has quit IRC | 04:17 | |
| *** vinbs_ is now known as vinbs | 04:17 | |
| *** boden has joined #openstack-infra | 04:17 | |
| *** vinbs_ has joined #openstack-infra | 04:19 | |
| *** gokrokve has joined #openstack-infra | 04:20 | |
| *** boden has quit IRC | 04:22 | |
| *** vinbs has quit IRC | 04:22 | |
| *** vinbs_ has quit IRC | 04:24 | |
| *** vinbs has joined #openstack-infra | 04:25 | |
| *** otherwiseguy has quit IRC | 04:26 | |
| openstackgerrit | patrick-crews proposed a change to openstack-infra/config: Adding tripleo-ci files to be indexed by logstash https://review.openstack.org/107841 | 04:26 |
| *** otherwiseguy has joined #openstack-infra | 04:26 | |
| pleia2 | clarkb: woo, I met nibalizer \o/ | 04:28 |
| *** lcheng_ has joined #openstack-infra | 04:29 | |
| *** vdreamarkitex has joined #openstack-infra | 04:29 | |
| *** lcheng has quit IRC | 04:31 | |
| *** yfried|afk has quit IRC | 04:32 | |
| *** achanda has quit IRC | 04:33 | |
| *** achanda has joined #openstack-infra | 04:34 | |
| *** rmcall has quit IRC | 04:34 | |
| *** pcrews has quit IRC | 04:34 | |
| *** signed8bit has quit IRC | 04:35 | |
| *** achanda_ has joined #openstack-infra | 04:36 | |
| *** tnovacik has quit IRC | 04:37 | |
| *** achanda has quit IRC | 04:38 | |
| *** radez is now known as radez_g0n3 | 04:39 | |
| *** vdreamarkitex has quit IRC | 04:49 | |
| *** gokrokve_ has joined #openstack-infra | 04:50 | |
| *** gokrokve has quit IRC | 04:53 | |
| *** MaxV has joined #openstack-infra | 04:58 | |
| nibalizer | wooo | 04:58 |
| nibalizer | pleia2: were drinking at tge top floor | 04:59 |
| *** yamahata has quit IRC | 04:59 | |
| *** yamahata has joined #openstack-infra | 05:00 | |
| *** bdpayne has quit IRC | 05:01 | |
| *** daya_k has joined #openstack-infra | 05:01 | |
| *** bdpayne has joined #openstack-infra | 05:02 | |
| *** Longgeek has quit IRC | 05:02 | |
| *** otherwiseguy has quit IRC | 05:06 | |
| pleia2 | nibalizer: nice view from that bar (when it's not foggy!) | 05:07 |
| pleia2 | but the pliny the elder double ipa is only at the downstairs bar :( | 05:07 |
| nibalizer | bodeco is buying..... | 05:07 |
| pleia2 | \o/ | 05:07 |
| jogo | have a few patches that could use some bumping | 05:08 |
| jogo | https://review.openstack.org/#/c/123021/4 https://review.openstack.org/#/c/123250 | 05:09 |
| jogo | https://review.openstack.org/#/c/123198 | 05:09 |
| jogo | are all for the top gate bug | 05:09 |
| jogo | 114 hits in 24 hours | 05:09 |
| jogo | oh 123198 is near the top already | 05:09 |
| jogo | well lets hope it doesn't bounce out | 05:10 |
| *** harlowja_at_home has joined #openstack-infra | 05:11 | |
| *** gokrokve_ has quit IRC | 05:13 | |
| *** gokrokve has joined #openstack-infra | 05:14 | |
| *** ChuckC_ has quit IRC | 05:15 | |
| *** Longgeek has joined #openstack-infra | 05:17 | |
| *** boden has joined #openstack-infra | 05:18 | |
| *** gokrokve has quit IRC | 05:18 | |
| *** achanda_ has quit IRC | 05:21 | |
| *** achanda has joined #openstack-infra | 05:21 | |
| *** harlowja_at_home has quit IRC | 05:21 | |
| *** boden has quit IRC | 05:22 | |
| *** achanda_ has joined #openstack-infra | 05:24 | |
| *** unicell has joined #openstack-infra | 05:24 | |
| *** bdpayne has quit IRC | 05:24 | |
| *** gokrokve has joined #openstack-infra | 05:24 | |
| *** ildikov has joined #openstack-infra | 05:25 | |
| *** vhoward has joined #openstack-infra | 05:26 | |
| *** achanda has quit IRC | 05:26 | |
| *** mrmartin has joined #openstack-infra | 05:26 | |
| *** hdd has quit IRC | 05:30 | |
| *** MaxV has quit IRC | 05:31 | |
| *** harlowja is now known as harlowja_away | 05:31 | |
| *** nadya_ has quit IRC | 05:41 | |
| *** ppai has joined #openstack-infra | 05:42 | |
| *** yfried|afk has joined #openstack-infra | 05:43 | |
| *** isaacb has joined #openstack-infra | 05:43 | |
| *** penguinRaider has joined #openstack-infra | 05:46 | |
| *** xyang1 has quit IRC | 05:47 | |
| *** SumitNaiksatam has quit IRC | 05:47 | |
| *** SumitNaiksatam has joined #openstack-infra | 05:48 | |
| *** shashankhegde has quit IRC | 05:48 | |
| *** SumitNaiksatam has quit IRC | 05:49 | |
| *** tnurlygayanov has quit IRC | 05:52 | |
| *** tnurlygayanov has joined #openstack-infra | 05:52 | |
| *** rushiagr_away is now known as rushiagr | 05:52 | |
| *** ilyashakhat has quit IRC | 05:53 | |
| *** afazekas has joined #openstack-infra | 05:54 | |
| *** ChuckC_ has joined #openstack-infra | 05:54 | |
| *** ilyashakhat has joined #openstack-infra | 05:55 | |
| *** tnurlygayanov has quit IRC | 05:56 | |
| *** tnurlygayanov has joined #openstack-infra | 05:56 | |
| *** SumitNaiksatam has joined #openstack-infra | 05:58 | |
| *** ilyashakhat has quit IRC | 06:00 | |
| *** morgan_remote_ has quit IRC | 06:00 | |
| *** ilyashakhat has joined #openstack-infra | 06:01 | |
| *** garyh has joined #openstack-infra | 06:02 | |
| *** flaper87|afk is now known as flaper87 | 06:05 | |
| *** ilyashakhat has quit IRC | 06:05 | |
| *** MaxV has joined #openstack-infra | 06:06 | |
| *** ilyashakhat has joined #openstack-infra | 06:06 | |
| *** [HeOS] has quit IRC | 06:06 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Initial centos7 support for build-image.sh https://review.openstack.org/123343 | 06:07 |
| *** tnurlygayanov has quit IRC | 06:08 | |
| *** jcoufal has joined #openstack-infra | 06:10 | |
| *** garyh has quit IRC | 06:12 | |
| *** tnurlygayanov has joined #openstack-infra | 06:12 | |
| *** ashaeron has joined #openstack-infra | 06:13 | |
| *** dimtruck is now known as zz_dimtruck | 06:13 | |
| *** isaacb has quit IRC | 06:17 | |
| *** boden has joined #openstack-infra | 06:18 | |
| *** MaxV has quit IRC | 06:20 | |
| *** MaxV has joined #openstack-infra | 06:22 | |
| *** ashaeron has quit IRC | 06:22 | |
| *** boden has quit IRC | 06:23 | |
| *** k4n0 has joined #openstack-infra | 06:23 | |
| *** ashaeron has joined #openstack-infra | 06:24 | |
| *** MaxV has quit IRC | 06:27 | |
| *** JadnG has quit IRC | 06:28 | |
| *** zns has quit IRC | 06:29 | |
| *** MaxV has joined #openstack-infra | 06:30 | |
| *** jeffrey4l has joined #openstack-infra | 06:31 | |
| *** andreaf has quit IRC | 06:32 | |
| *** andreaf has joined #openstack-infra | 06:32 | |
| jeffrey4l | Any guys can review my patch for add keystone-formula: https://review.openstack.org/#/c/120324/ | 06:32 |
| *** loki184 has joined #openstack-infra | 06:33 | |
| *** lcheng_ has quit IRC | 06:37 | |
| *** JadnG has joined #openstack-infra | 06:41 | |
| *** stevemar has quit IRC | 06:43 | |
| *** achanda_ has quit IRC | 06:43 | |
| *** penguinRaider has quit IRC | 06:44 | |
| mikal | mordred: you around? | 06:45 |
| mordred | mikal: just went to bed | 06:45 |
| mikal | mordred: LOL, ok. Nothing that urgent. | 06:46 |
| mikal | mordred: just your marketing department freaking me out | 06:46 |
| AJaeger | SergeyLukjanov, regarding publishing python-manilaclient on pypi: Who creates the project on pypi? Is that an infra job, one for PTL - or can I do this? | 06:46 |
| AJaeger | mordred, good night! | 06:46 |
| *** mrmartin has quit IRC | 06:46 | |
| *** gokrokve_ has joined #openstack-infra | 06:49 | |
| *** isaacb has joined #openstack-infra | 06:51 | |
| *** gokrokve has quit IRC | 06:51 | |
| *** gokrokve_ has quit IRC | 06:54 | |
| *** rdopieralski has joined #openstack-infra | 06:56 | |
| *** michchap has quit IRC | 06:56 | |
| *** mpaolino has joined #openstack-infra | 07:01 | |
| *** MaxV has quit IRC | 07:03 | |
| *** sdake has quit IRC | 07:06 | |
| *** morgan_remote_ has joined #openstack-infra | 07:09 | |
| *** andreaf has quit IRC | 07:11 | |
| *** michchap has joined #openstack-infra | 07:12 | |
| *** jcoufal has quit IRC | 07:18 | |
| *** nadya_ has joined #openstack-infra | 07:19 | |
| *** pblaho has joined #openstack-infra | 07:20 | |
| *** jcoufal has joined #openstack-infra | 07:20 | |
| *** gokrokve has joined #openstack-infra | 07:25 | |
| *** gokrokve has quit IRC | 07:27 | |
| *** gokrokve has joined #openstack-infra | 07:27 | |
| *** ihrachyshka has joined #openstack-infra | 07:27 | |
| *** arxcruz has joined #openstack-infra | 07:29 | |
| *** reed has quit IRC | 07:29 | |
| *** jp_at_hp has joined #openstack-infra | 07:30 | |
| *** salv-orlando has joined #openstack-infra | 07:31 | |
| *** MarkAtwood has quit IRC | 07:31 | |
| *** gokrokve has quit IRC | 07:32 | |
| *** praneshp_ has joined #openstack-infra | 07:32 | |
| *** praneshp has quit IRC | 07:32 | |
| *** praneshp_ is now known as praneshp | 07:32 | |
| *** marcoemorais1 has quit IRC | 07:33 | |
| openstackgerrit | Prashanth Pai proposed a change to openstack-infra/config: swiftonfile: Add py26 https://review.openstack.org/123378 | 07:33 |
| *** [HeOS] has joined #openstack-infra | 07:36 | |
| *** MaxV has joined #openstack-infra | 07:36 | |
| *** jeffrey4l has quit IRC | 07:36 | |
| *** [HeOS] is now known as HeOS | 07:42 | |
| *** jeffrey4l has joined #openstack-infra | 07:44 | |
| *** SumitNaiksatam has quit IRC | 07:45 | |
| *** derekh has joined #openstack-infra | 07:47 | |
| *** che-arne has quit IRC | 07:49 | |
| *** Ryan_Lane has quit IRC | 07:50 | |
| *** e0ne has joined #openstack-infra | 07:50 | |
| *** berendt has joined #openstack-infra | 07:52 | |
| *** unicell has quit IRC | 07:52 | |
| *** ociuhandu has quit IRC | 07:54 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 07:57 | |
| *** e0ne has quit IRC | 07:59 | |
| *** unicell has joined #openstack-infra | 07:59 | |
| *** boden has joined #openstack-infra | 08:00 | |
| *** sdake has joined #openstack-infra | 08:00 | |
| *** e0ne has joined #openstack-infra | 08:00 | |
| *** praneshp has quit IRC | 08:01 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Switch to static content for glance upload test https://review.openstack.org/122727 | 08:01 |
| *** hashar has joined #openstack-infra | 08:02 | |
| *** andreykurilin has joined #openstack-infra | 08:03 | |
| *** jistr has joined #openstack-infra | 08:04 | |
| *** Hal has joined #openstack-infra | 08:04 | |
| *** Hal is now known as Guest34961 | 08:05 | |
| *** vdo has joined #openstack-infra | 08:06 | |
| *** Longgeek has quit IRC | 08:07 | |
| *** liusheng has joined #openstack-infra | 08:13 | |
| *** ildikov has quit IRC | 08:13 | |
| *** mpaolino has quit IRC | 08:14 | |
| *** denis_makogon has joined #openstack-infra | 08:17 | |
| *** jgallard has joined #openstack-infra | 08:22 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Remove tripleo control variables https://review.openstack.org/122504 | 08:25 |
| *** gokrokve has joined #openstack-infra | 08:26 | |
| *** Longgeek has joined #openstack-infra | 08:29 | |
| *** gokrokve has quit IRC | 08:31 | |
| *** penguinRaider has joined #openstack-infra | 08:31 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/config: Add experimental zaqar-rally job https://review.openstack.org/123396 | 08:35 |
| *** amuller has joined #openstack-infra | 08:38 | |
| *** mkerrin has quit IRC | 08:41 | |
| *** nadya_ has quit IRC | 08:43 | |
| *** kim__ has joined #openstack-infra | 08:47 | |
| kim__ | I need some help tocopy logs... | 08:47 |
| kim__ | Thanks | 08:47 |
| *** p5ntangle has joined #openstack-infra | 08:54 | |
| *** e0ne has quit IRC | 08:58 | |
| *** andreykurilin has quit IRC | 08:58 | |
| *** jcoufal has quit IRC | 08:58 | |
| *** jlibosva has joined #openstack-infra | 08:59 | |
| *** IvanBerezovskiy has joined #openstack-infra | 09:03 | |
| JadnG | Hello, Fungi , could you pls add initial Gerrit group for Tricircle project. please refer to https://bugs.launchpad.net/openstack-ci/+bug/1363860 . thanks a lot. | 09:05 |
| uvirtbot | Launchpad bug 1363860 in openstack-ci "request gerrit group for tricircle" [Undecided,New] | 09:05 |
| *** jgallard has quit IRC | 09:08 | |
| *** e0ne has joined #openstack-infra | 09:09 | |
| *** morgan_remote_ has quit IRC | 09:10 | |
| *** mkerrin has joined #openstack-infra | 09:11 | |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/config: Solve bashate issue E011 https://review.openstack.org/122687 | 09:13 |
| openstackgerrit | Christian Berendt proposed a change to openstack-infra/config: Solve bashate issues E010 and E004 https://review.openstack.org/122688 | 09:13 |
| *** dizquierdo has joined #openstack-infra | 09:14 | |
| *** jcoufal has joined #openstack-infra | 09:15 | |
| *** tnovacik has joined #openstack-infra | 09:17 | |
| *** KanagarajM has joined #openstack-infra | 09:26 | |
| *** bhuvan has quit IRC | 09:26 | |
| *** gokrokve has joined #openstack-infra | 09:26 | |
| *** JadnG has quit IRC | 09:30 | |
| *** gokrokve has quit IRC | 09:30 | |
| *** rlandy has joined #openstack-infra | 09:32 | |
| *** cnesa14 has joined #openstack-infra | 09:36 | |
| *** hashar_ has joined #openstack-infra | 09:36 | |
| *** andreaf_ is now known as andreaf | 09:36 | |
| *** hashar has quit IRC | 09:36 | |
| *** jeffrey4l has quit IRC | 09:40 | |
| *** tnovacik has quit IRC | 09:42 | |
| *** kim__ has quit IRC | 09:44 | |
| *** JadnG has joined #openstack-infra | 09:47 | |
| *** yfried|afk has quit IRC | 09:50 | |
| *** yfried has joined #openstack-infra | 09:50 | |
| *** yfried is now known as yfried|afk | 09:50 | |
| *** jeffrey4l has joined #openstack-infra | 09:53 | |
| *** flaviof_zzz has quit IRC | 09:53 | |
| *** yfried has joined #openstack-infra | 09:53 | |
| *** sarob has quit IRC | 09:53 | |
| *** che-arne has joined #openstack-infra | 09:55 | |
| *** yfried|afk has quit IRC | 09:55 | |
| *** ildikov has joined #openstack-infra | 09:57 | |
| *** yfried has quit IRC | 09:58 | |
| *** daya_k has quit IRC | 09:59 | |
| *** ociuhandu has joined #openstack-infra | 10:03 | |
| *** Guest34961 has quit IRC | 10:07 | |
| *** jpich has quit IRC | 10:12 | |
| *** JadnG has quit IRC | 10:12 | |
| *** p5ntangl_ has joined #openstack-infra | 10:17 | |
| *** p5ntangl_ has quit IRC | 10:17 | |
| *** p5ntangl_ has joined #openstack-infra | 10:18 | |
| *** xchu has quit IRC | 10:20 | |
| *** p5ntangle has quit IRC | 10:20 | |
| *** pcm_ has joined #openstack-infra | 10:21 | |
| *** p5ntangl_ has quit IRC | 10:21 | |
| *** p5ntangle has joined #openstack-infra | 10:22 | |
| *** pcm_ has quit IRC | 10:22 | |
| *** pcm_ has joined #openstack-infra | 10:23 | |
| *** nadya has joined #openstack-infra | 10:23 | |
| *** nadya is now known as Guest81122 | 10:23 | |
| *** e0ne has quit IRC | 10:23 | |
| *** ppai has quit IRC | 10:25 | |
| *** gokrokve has joined #openstack-infra | 10:26 | |
| *** JadnG has joined #openstack-infra | 10:28 | |
| *** hashar_ has quit IRC | 10:30 | |
| *** gokrokve has quit IRC | 10:30 | |
| *** flaviof_zzz has joined #openstack-infra | 10:31 | |
| *** ppai has joined #openstack-infra | 10:37 | |
| *** yaguang has quit IRC | 10:42 | |
| *** e0ne has joined #openstack-infra | 10:46 | |
| *** koolhead17 has quit IRC | 10:50 | |
| *** Longgeek has quit IRC | 10:50 | |
| *** yfried has joined #openstack-infra | 10:52 | |
| *** yfried is now known as yfried|afk | 10:52 | |
| *** isaacb has quit IRC | 10:54 | |
| *** mkoderer_ is now known as mkoderer | 10:55 | |
| *** p5ntangle has quit IRC | 11:00 | |
| *** p5ntangle has joined #openstack-infra | 11:01 | |
| *** QuinnyPig has quit IRC | 11:01 | |
| *** QuinnyPig has joined #openstack-infra | 11:01 | |
| *** Longgeek has joined #openstack-infra | 11:03 | |
| *** jyuso has quit IRC | 11:04 | |
| *** p5ntangle has quit IRC | 11:07 | |
| *** p5ntangle has joined #openstack-infra | 11:08 | |
| *** p5ntangle has joined #openstack-infra | 11:08 | |
| *** miarmak has joined #openstack-infra | 11:12 | |
| *** vdreamarkitex has joined #openstack-infra | 11:17 | |
| *** dmakogon_ has joined #openstack-infra | 11:18 | |
| *** denis_makogon has quit IRC | 11:19 | |
| *** dmakogon_ is now known as denis_makogon | 11:19 | |
| *** denis_makogon_ has joined #openstack-infra | 11:19 | |
| *** pcm_ has quit IRC | 11:19 | |
| *** ppai has quit IRC | 11:20 | |
| *** dmsimard_away is now known as dmsimard | 11:23 | |
| *** dims has joined #openstack-infra | 11:25 | |
| *** gokrokve has joined #openstack-infra | 11:26 | |
| *** sarob has joined #openstack-infra | 11:29 | |
| *** gokrokve has quit IRC | 11:31 | |
| *** ppai has joined #openstack-infra | 11:33 | |
| *** hashar has joined #openstack-infra | 11:33 | |
| *** p5ntangl_ has joined #openstack-infra | 11:33 | |
| *** loki184 has quit IRC | 11:34 | |
| *** isviridov_away is now known as isviridov | 11:35 | |
| *** p5ntangle has quit IRC | 11:37 | |
| *** eglynn-officeafk is now known as eglynn-office | 11:41 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/config: Add rally-designate experimental job https://review.openstack.org/123421 | 11:41 |
| *** Guest81122 has quit IRC | 11:42 | |
| *** p5ntangl_ has quit IRC | 11:45 | |
| *** p5ntangle has joined #openstack-infra | 11:46 | |
| *** flaviof_zzz is now known as flaviof | 11:46 | |
| *** lttrl has joined #openstack-infra | 11:47 | |
| *** berendt has quit IRC | 11:48 | |
| *** nadya has joined #openstack-infra | 11:48 | |
| *** nadya is now known as Guest59271 | 11:48 | |
| *** Longgeek has quit IRC | 11:52 | |
| *** kgiusti has joined #openstack-infra | 11:52 | |
| *** ppai has quit IRC | 11:53 | |
| sdague | who are the experts on the toggle ci code? | 11:54 |
| *** p5ntangle has quit IRC | 11:56 | |
| *** weshay has joined #openstack-infra | 11:56 | |
| *** p5ntangle has joined #openstack-infra | 11:56 | |
| sdague | great, the rackspace deb mirrors are completely fubared | 11:57 |
| *** mwagner_lap has quit IRC | 11:59 | |
| *** eglynn-office is now known as eglynn-lunch | 11:59 | |
| *** Longgeek has joined #openstack-infra | 12:00 | |
| *** Longgeek has quit IRC | 12:00 | |
| *** Longgeek has joined #openstack-infra | 12:00 | |
| *** bswartz has quit IRC | 12:03 | |
| *** mbacchi has joined #openstack-infra | 12:05 | |
| *** julim has joined #openstack-infra | 12:07 | |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/config: Introduce tripleo-puppet-elements into Stackforge https://review.openstack.org/120285 | 12:08 |
| *** k4n0 has quit IRC | 12:09 | |
| EmilienM | jhesketh: SergeyLukjanov: just a rebase, could you please +2 again? ^^^^ | 12:09 |
| *** ppai has joined #openstack-infra | 12:10 | |
| *** yfried|afk is now known as yfried | 12:11 | |
| *** dims has quit IRC | 12:13 | |
| *** dims has joined #openstack-infra | 12:14 | |
| *** signed8bit has joined #openstack-infra | 12:14 | |
| *** pcm_ has joined #openstack-infra | 12:16 | |
| *** pcm_ has quit IRC | 12:16 | |
| *** pcm_ has joined #openstack-infra | 12:17 | |
| *** vinbs has quit IRC | 12:17 | |
| *** zns has joined #openstack-infra | 12:17 | |
| *** vinbs has joined #openstack-infra | 12:17 | |
| *** vinbs has quit IRC | 12:18 | |
| *** vinbs has joined #openstack-infra | 12:18 | |
| *** zns has quit IRC | 12:19 | |
| *** dizquierdo is now known as dizquierdo_afk | 12:19 | |
| *** yfried is now known as yfried|afk | 12:21 | |
| *** yfried|afk is now known as yfried | 12:24 | |
| *** dkranz has joined #openstack-infra | 12:25 | |
| *** acfleury has joined #openstack-infra | 12:25 | |
| *** mmedvede has joined #openstack-infra | 12:26 | |
| *** gokrokve has joined #openstack-infra | 12:26 | |
| *** jaypipes has joined #openstack-infra | 12:27 | |
| *** hashar has quit IRC | 12:28 | |
| *** KanagarajM has quit IRC | 12:29 | |
| *** gokrokve has quit IRC | 12:31 | |
| *** dmellado has joined #openstack-infra | 12:32 | |
| *** yfried is now known as yfried|afk | 12:38 | |
| *** koolhead17 has joined #openstack-infra | 12:38 | |
| *** vponomaryov has quit IRC | 12:38 | |
| *** funzo_ is now known as funzo | 12:39 | |
| *** aysyanne has joined #openstack-infra | 12:39 | |
| *** hashar has joined #openstack-infra | 12:41 | |
| *** vponomaryov has joined #openstack-infra | 12:41 | |
| *** Guest34961 has joined #openstack-infra | 12:41 | |
| *** dprince has joined #openstack-infra | 12:42 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: Remove lesscpy https://review.openstack.org/122304 | 12:42 |
| *** yfried|afk is now known as yfried | 12:42 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: Require kombu 2.5.0 or newer for switch to py-amqp https://review.openstack.org/92095 | 12:47 |
| *** mjturek has joined #openstack-infra | 12:50 | |
| *** bswartz has joined #openstack-infra | 12:50 | |
| *** sabeen1 has quit IRC | 12:50 | |
| *** weshay has quit IRC | 12:51 | |
| *** jcoufal has quit IRC | 12:52 | |
| *** jcoufal has joined #openstack-infra | 12:55 | |
| *** lttrl has quit IRC | 12:55 | |
| *** masayukig has quit IRC | 12:58 | |
| *** baoli has joined #openstack-infra | 12:58 | |
| *** pblaho has quit IRC | 12:58 | |
| *** pblaho has joined #openstack-infra | 13:00 | |
| *** xyang0 has joined #openstack-infra | 13:01 | |
| *** arxcruz has quit IRC | 13:02 | |
| *** koolhead17 has quit IRC | 13:02 | |
| *** masayukig has joined #openstack-infra | 13:03 | |
| *** weshay has joined #openstack-infra | 13:05 | |
| *** ppai has quit IRC | 13:05 | |
| *** mjturek has quit IRC | 13:07 | |
| *** mjturek has joined #openstack-infra | 13:08 | |
| *** mjturek has quit IRC | 13:09 | |
| *** vinbs has quit IRC | 13:09 | |
| fungi | ugh. internet service at my house is out this morning. working from a payg wireless modem for now | 13:11 |
| dims | fungi: ouch | 13:12 |
| fungi | nodepool seems to have survived the night | 13:12 |
| *** juzuluag has joined #openstack-infra | 13:12 | |
| fungi | sdague: are rax package mirrors still busted, or has that cleared up since your comment? | 13:13 |
| *** changbl_ has quit IRC | 13:13 | |
| ZZelle | fungi, hi | 13:14 |
| fungi | ZZelle: hi there--fixing something? | 13:15 |
| *** miqui has joined #openstack-infra | 13:15 | |
| *** arxcruz has joined #openstack-infra | 13:15 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: Correcting case of pyscss to match pypi https://review.openstack.org/117910 | 13:15 |
| ZZelle | fungi, nope, just to reminder to look at git-review changes in gerrit :) | 13:15 |
| * fungi nods | 13:16 | |
| *** mjturek has joined #openstack-infra | 13:16 | |
| *** jpeeler has joined #openstack-infra | 13:16 | |
| *** zns has joined #openstack-infra | 13:17 | |
| ZZelle | fungi, thanks | 13:18 |
| *** zns has quit IRC | 13:19 | |
| *** mriedem has joined #openstack-infra | 13:19 | |
| *** penguinRaider has quit IRC | 13:19 | |
| *** jgallard has joined #openstack-infra | 13:20 | |
| *** pcm_ has quit IRC | 13:21 | |
| *** zns has joined #openstack-infra | 13:23 | |
| *** dustins has joined #openstack-infra | 13:23 | |
| *** signed8bit has quit IRC | 13:24 | |
| *** signed8bit has joined #openstack-infra | 13:25 | |
| *** eglynn-lunch is now known as eglynn | 13:26 | |
| *** mfer has joined #openstack-infra | 13:26 | |
| *** gokrokve has joined #openstack-infra | 13:26 | |
| *** bknudson has joined #openstack-infra | 13:28 | |
| HeOS | Hi to all! I wonder how you added jenkins status in gerrit such as in this ticket https://review.fuel-infra.org/16. Is the status a default feature in gerrit? | 13:28 |
| *** SumitNaiksatam has joined #openstack-infra | 13:28 | |
| HeOS | Hm. | 13:29 |
| HeOS | Sorry. | 13:29 |
| HeOS | This ticket: https://review.openstack.org/#/c/120377/ | 13:29 |
| fungi | HeOS: what do you mean by jenkins status? | 13:30 |
| *** funzo has quit IRC | 13:30 | |
| *** Guest59271 has quit IRC | 13:31 | |
| *** jeffrey4l has left #openstack-infra | 13:31 | |
| *** gokrokve has quit IRC | 13:31 | |
| *** radez_g0n3 is now known as radez | 13:32 | |
| fungi | HeOS: i think you might be referring to https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/gerrit/hideci.js | 13:32 |
| HeOS | For example "gate-nova-pep8SUCCESS in 9m 56s" in right panel with name 'Jenkins' | 13:32 |
| fungi | HeOS: right, that's being done by the javascript i linked above | 13:33 |
| *** stevemar has joined #openstack-infra | 13:34 | |
| *** vhoward has left #openstack-infra | 13:35 | |
| HeOS | fungi, thanks. :) | 13:35 |
| HeOS | fungi, should I add this script in gerrit? How this script work with gerri? | 13:38 |
| HeOS | *gerrit | 13:38 |
| fungi | HeOS: we use a custom gerrit site header to include it... https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/gerrit/GerritSiteHeader.html | 13:38 |
| *** andreaf is now known as andreaf_ | 13:38 | |
| *** sabeen1 has joined #openstack-infra | 13:39 | |
| *** zz_jgrimm is now known as jgrimm | 13:40 | |
| *** dizquierdo_afk is now known as dizquierdo | 13:43 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Merger: ensure_cloned() now looks for '.git' https://review.openstack.org/123437 | 13:44 |
| *** dustins has quit IRC | 13:45 | |
| *** armax has joined #openstack-infra | 13:45 | |
| *** daya_k has joined #openstack-infra | 13:46 | |
| *** pblaho has quit IRC | 13:46 | |
| *** ildikov_ has joined #openstack-infra | 13:46 | |
| rdopieralski | sdague: about https://review.openstack.org/#/c/121509/ that other bug got fixed in pyscss after all, so we are fine with 1.0.3 | 13:46 |
| rdopieralski | sdague: sorry about not being clear about it | 13:46 |
| *** dustins has joined #openstack-infra | 13:46 | |
| *** pblaho has joined #openstack-infra | 13:48 | |
| HeOS | fungi, How this header add to gerrit, just as at this article? http://gerrit.googlecode.com/svn/documentation/2.1/config-headerfooter.html | 13:48 |
| *** garyh has joined #openstack-infra | 13:48 | |
| fungi | HeOS: we use puppet to manage our gerrit server. i recommend reviewing https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/review.pp and the other classes/modules it calls | 13:50 |
| sdague | rdopieralski: ok, I think it was going to go to the release meeting today, or poke ttx on it | 13:50 |
| fungi | HeOS: also http://ci.openstack.org/gerrit.html | 13:50 |
| *** ildikov has quit IRC | 13:50 | |
| rdopieralski | ttx: poke, poke :) | 13:50 |
| *** markmcclain has joined #openstack-infra | 13:50 | |
| *** mwagner_lap has joined #openstack-infra | 13:50 | |
| hashar | HeOS: in gerrit, the 'Jenkins' right panel is some openstack custom javascript | 13:51 |
| sdague | fungi: yeh, mirrors are still destroying us - http://logs.openstack.org/44/122644/2/gate/gate-devstack-dsvm-cells/8755c8f/logs/devstacklog.txt.gz#_2014-09-23_13_35_02_809 | 13:51 |
| hashar | HeOS: definitely not build in gerrit | 13:51 |
| sdague | It would be really nice if rackspace had functional mirrors | 13:51 |
| *** markmcclain1 has joined #openstack-infra | 13:52 | |
| fungi | sdague: i have a half-baked change to devstack which would let us pass in alternate package mirror urls and then fall back on that if a package install attempt fails... i haven't spent much more time on it because it was unclear whether that added complexity was worth it/would be welcome in devstack (which is really the only place to do it) | 13:53 |
| *** markmcclain1 has quit IRC | 13:53 | |
| fungi | and the frequency of that issue had trailed waaaay off | 13:53 |
| *** otherwiseguy has joined #openstack-infra | 13:54 | |
| jroll | I don't want to do it right this minute, but is there a way to get into a nodepool node to debug something that might be environment-specific? | 13:54 |
| jroll | looking at http://logs.openstack.org/60/122360/1/check/check-tempest-dsvm-ironic-agent_ssh-nv/69e91e9/logs/ironic-bm-logs/baremetalbrbm_0_console.txt.gz | 13:54 |
| jroll | has always worked with devstack on a rackspace cloud server, so I have no idea where to start | 13:54 |
| *** markmcclain has quit IRC | 13:55 | |
| ttx | rdopieralski: o/ | 13:55 |
| *** lcheng has joined #openstack-infra | 13:55 | |
| *** pcm_ has joined #openstack-infra | 13:55 | |
| ttx | sdague: maybe remove you Workflow-1 from https://review.openstack.org/#/c/121509/ -- and we'll discuss it at meeting today | 13:56 |
| fungi | also i'm concerned that adding fallback mirrors would just hide/shift the problem so that we wouldn't notice when some major change to their mirroring infrastructure permanently broke the primary mirror and we'd indefinitely rely on a less efficient fallback (until it too started breaking at random and bringing back the same symptoms) | 13:56 |
| *** markmcclain has joined #openstack-infra | 13:56 | |
| sdague | ttx: done | 13:56 |
| sdague | yeh, I'm just incredibly frustrated that rax deb mirroring is so broken | 13:57 |
| *** nelsnelson has joined #openstack-infra | 13:57 | |
| *** lcheng has quit IRC | 13:57 | |
| sdague | because if you do these things in the right order, you won't break the freaking world if we hit you during an in process update | 13:57 |
| *** dustins has quit IRC | 13:58 | |
| *** marun has joined #openstack-infra | 13:58 | |
| mordred | actually, it's a general problem with apt repos | 13:58 |
| mordred | it's not possibly to atomically update them | 13:59 |
| *** r-daneel_ has joined #openstack-infra | 14:00 | |
| *** homeless has joined #openstack-infra | 14:00 | |
| *** weshay has quit IRC | 14:00 | |
| *** nadya has joined #openstack-infra | 14:01 | |
| *** signed8bit has quit IRC | 14:01 | |
| *** nadya is now known as Guest13012 | 14:01 | |
| *** dane_leblanc has joined #openstack-infra | 14:02 | |
| sdague | sigh | 14:02 |
| *** dustins has joined #openstack-infra | 14:03 | |
| *** yfried is now known as yfried|afk | 14:03 | |
| hashar | well you could use two and switch from outdated to updated | 14:04 |
| sdague | yeh, well, regardless, everything is fail at the moment because of it | 14:04 |
| hashar | apt1 , apt2 . apt points to apt1, update apt2 and once done switch apt to now updated apt2 | 14:04 |
| hashar | but that is annoying for usre | 14:04 |
| *** gokrokve_ has joined #openstack-infra | 14:05 | |
| sdague | it looks like one of the mirror.rackspace.com is garbage based on the frequency | 14:05 |
| *** mflobo_ has joined #openstack-infra | 14:05 | |
| sdague | there are 6 in the dns round robin | 14:05 |
| *** willroberts_ has quit IRC | 14:06 | |
| *** willroberts_ has joined #openstack-infra | 14:06 | |
| *** dkranz has quit IRC | 14:06 | |
| *** sweston has quit IRC | 14:06 | |
| *** mflobo has quit IRC | 14:07 | |
| mriedem | should we be concerned about the logstash job queue? | 14:07 |
| *** zz_gondoi is now known as gondoi | 14:07 | |
| mriedem | it hasn't dipped below 30K in awhile | 14:07 |
| *** sweston has joined #openstack-infra | 14:08 | |
| *** Guest13012 has quit IRC | 14:10 | |
| sdague | yeh, would be good to get under control again. Hard to address top bugs when we are lagging by 12+ hrs | 14:11 |
| mordred | do we need to make an apt mirror? | 14:12 |
| *** mflobo_ has quit IRC | 14:13 | |
| sdague | mordred: or have someone from rax actually in channel here that can help fix | 14:13 |
| sdague | because I'm sure this is breaking lots of people | 14:13 |
| *** mflobo has joined #openstack-infra | 14:13 | |
| mordred | phschwartz: ^^^ | 14:13 |
| *** weshay has joined #openstack-infra | 14:13 | |
| *** cnesa15 has joined #openstack-infra | 14:14 | |
| jroll | hi | 14:14 |
| *** cnesa14 has quit IRC | 14:14 | |
| jroll | broken mirror? | 14:14 |
| jroll | sdague: any chance you can pinpoint which one? | 14:14 |
| *** baoli has quit IRC | 14:15 | |
| mfer | Folks, if there is an external tool we could link up via github webhook is there a process to do so? | 14:16 |
| jroll | (yes, this is a crappy problem we see often :( ) | 14:16 |
| jroll | mfer: what sort of thing? openstack only uses github as a mirror, we have other tools for post-merge hooks and whatnot | 14:17 |
| mfer | some things only work with Github. For example, packagist.org for the PHP SDK | 14:17 |
| mfer | it's not going to work from general git | 14:17 |
| jroll | I see | 14:18 |
| jroll | mordred, sdague, if y'all can pinpoint which mirror I can poke people | 14:18 |
| sdague | jroll: I have no idea which mirror | 14:19 |
| jroll | ok | 14:19 |
| * jroll finds people | 14:19 | |
| sdague | I just know at least 1 of them has a version of nspr4 in their metadata that they don't have in their repo | 14:19 |
| sdague | http://logs.openstack.org/26/123426/1/check/check-tempest-dsvm-neutron-heat-slow/7c2dc94/logs/devstacklog.txt.gz | 14:20 |
| sdague | and it's basically preventing any openstack patches from progressing in zuul | 14:20 |
| jroll | locely | 14:20 |
| jroll | lovely, even | 14:20 |
| *** david-lyle has joined #openstack-infra | 14:21 | |
| sdague | yeh, if a rackspace mirror is broken, it basically stops openstack development | 14:21 |
| sdague | http://status.openstack.org/zuul/ - note the fun red :) | 14:21 |
| *** doug-fish has joined #openstack-infra | 14:21 | |
| jroll | I'm aware, it also breaks internal things for our team so I have a vested interest :) | 14:21 |
| *** dangers_away is now known as dangers | 14:22 | |
| *** homeless has quit IRC | 14:22 | |
| mordred | mfer: no, there is not currently a way to do that. let me think about it though and get back to you | 14:22 |
| *** homeless has joined #openstack-infra | 14:23 | |
| mfer | mordred thanks. this is just one example. there are lots of tools designed to leverage github. some will also take bitbucket and gitlab.com. If we have the mirrors it would be nice to leverage them for some other things | 14:23 |
| jroll | sdague: folks are on it, will let you know | 14:24 |
| * mfer is a pragmatist and likes to leverage the things | 14:24 | |
| HeOS | fungi, hashar thank you, I understand how it works. :) | 14:25 |
| mordred | mfer: well, I understand the sentiment, but we aren't not doing things with github because of purity of thought, and the overall cost to be able to depend on unreliable tools is high | 14:25 |
| *** zz_dimtruck is now known as dimtruck | 14:25 | |
| mfer | mordred what about using tools on the perifery that's aren't reliable but we don't rely on them. instead they are useful and help extend openstacks reach. | 14:26 |
| *** dimtruck is now known as zz_dimtruck | 14:26 | |
| mordred | mfer: we operate at twice the volume of all of Travis ci here ... things that work for others often do not work for us | 14:26 |
| *** baoli has joined #openstack-infra | 14:26 | |
| *** zz_dimtruck is now known as dimtruck | 14:26 | |
| mfer | mordred i totally get that. it's not for everything | 14:27 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/requirements: Bump websockify to >=0.6.0 which fixes bug causing zombie processes https://review.openstack.org/114757 | 14:27 |
| mordred | mfer: right. that's why I'm going to think about it ... Just a heads up that it may not have a good solution | 14:27 |
| AJaeger | sdague, regarding https://review.openstack.org/122955 - do you have some time to discuss how to move forward here? | 14:28 |
| ihrachyshka | sdague: is it worth rechecking those failures due to apt mirror broken? | 14:28 |
| sdague | AJaeger: this really is a summit session on requirements role | 14:28 |
| sdague | ihrachyshka: probably not | 14:29 |
| sdague | from what I can see with 1 in 6 mirrors broken, the odds of a change passing is probably < 20% | 14:29 |
| AJaeger | sdague, fine with me - I'll attend ;) | 14:29 |
| ihrachyshka | sdague: yeah, lots of reviews are failing, so I doubt it's easy to pass check+verify | 14:29 |
| sdague | even if it didn't hit one of our bugs | 14:29 |
| *** jcoufal has quit IRC | 14:30 | |
| denis_makogon | zigo, ping | 14:33 |
| *** groknix has joined #openstack-infra | 14:35 | |
| *** yfried|afk is now known as yfried | 14:37 | |
| clarkb | mordred mfer I am not sure I understand the discussion. I s this a question of pointing additional tooling at stackforge or?? | 14:38 |
| *** signed8bit has joined #openstack-infra | 14:38 | |
| clarkb | sdague mriedem ya after I found the process that was bit bucketing logs the queue jumped back up again. I should probably continue the ssd volume stuff | 14:39 |
| jroll | clarkb: how much disk space do y'all need for ES? | 14:41 |
| mfer | clarkb my use cases are not about providing additional tooling to the openstack infrastructure. it's more about the openstack projects playing nicely with other systems and communities. I use packagist as an example because we have a PHP SDK and that would enable it to play nicely in that community. | 14:41 |
| clarkb | jroll right now its 6TB for 10 days and that handles a node dying | 14:42 |
| clarkb | jroll idealy we probably double that and reextend our backlog | 14:42 |
| jroll | hmmm | 14:42 |
| jroll | clarkb: we have some bare metal nodes with really fast disks.... | 14:42 |
| clarkb | jroll ya bare metal would be awesome :) | 14:43 |
| jroll | idk what the budget is like, they're ~$2k/mo, 3.6 TB or something? | 14:43 |
| jroll | clarkb: I can give you access to one for free for a bit, if you want to benchmark, I don't know that I have the power to give them indefinitely | 14:44 |
| jroll | clarkb: but you have to promise me a blog post with benchmarks :P | 14:44 |
| clarkb | jroll that is tempting but not right now. need to get nodepool updated and using dib first | 14:45 |
| *** gargola has joined #openstack-infra | 14:45 | |
| *** lcheng has joined #openstack-infra | 14:45 | |
| clarkb | when I can context switch I will ping you? | 14:45 |
| jroll | clarkb: heh, let me know :) | 14:45 |
| jroll | totally | 14:45 |
| mmedvede | asselin, regarding the https://review.openstack.org/#/c/122896/ - the hook calls could be moved out into functions.sh. But before that can happen, pre_test_hook call needs to be brought to the same form as the rest. There could be side effects of doing so, and it would take time to investigate (e.g. why we do not fail if the pre_test_hook fails, but we do if the post_test_hook does). | 14:45 |
| jroll | clarkb: we're really excited to get folks using these and talking about them | 14:46 |
| *** otherwiseguy has quit IRC | 14:46 | |
| jroll | and I think elastic search is a great use case | 14:46 |
| *** e0ne has quit IRC | 14:46 | |
| *** isviridov is now known as isviridov_meetin | 14:46 | |
| clarkb | mfer a quick read on packagist (and I am a noob here so possibly completely wrong) is that it crawls a public repo url. shouldnt that work with our repos just fine? | 14:48 |
| mfer | it crawls it from time to time. when setup via a webhook it does it on update | 14:49 |
| mfer | other services don't even go that far | 14:49 |
| fungi | also, sounds like a security nightmare (like pypi is working to undo now, constrained by historical baggage from previously supporting that use case) | 14:49 |
| phschwartz | sdague: What can I help with? I am one of infra's points of contact and help in Rax. | 14:51 |
| *** andreaf has joined #openstack-infra | 14:51 | |
| fungi | phschwartz: http://logs.openstack.org/26/123426/1/check/check-tempest-dsvm-neutron-heat-slow/7c2dc94/logs/devstacklog.txt.gz#_2014-09-23_12_55_33_906 | 14:52 |
| *** e0ne has joined #openstack-infra | 14:52 | |
| *** tonytan4ever has joined #openstack-infra | 14:52 | |
| fungi | er, actually earlier in that log, http://logs.openstack.org/26/123426/1/check/check-tempest-dsvm-neutron-heat-slow/7c2dc94/logs/devstacklog.txt.gz#_2014-09-23_12_55_27_965 | 14:52 |
| fungi | it tries a couple times, just to be sure the mirror is broken | 14:53 |
| phschwartz | Pingging the guys about the mirror | 14:53 |
| fungi | phschwartz: thanks! | 14:54 |
| *** ashaeron has quit IRC | 14:54 | |
| *** yolanda has joined #openstack-infra | 14:54 | |
| clarkb | mordred fungi apparently reprepro does all kinds of consistency checks before updating the live mirror? | 14:54 |
| *** reed has joined #openstack-infra | 14:54 | |
| fungi | i'm tempted to write an apt repo sanity checker, since if we start running a mirror of our own we'll need one to use as an acceptance test between pulses | 14:55 |
| clarkb | but ya the simple rsync tools can bite hard | 14:55 |
| *** hashar has quit IRC | 14:55 | |
| fungi | and yeah, supposedly if you download packages first and then download each index and its associated detached signatures in as close a proximity to one another as possible (or pivot them on links) then it's supposed to be basically transparent. the problem is many operators mirror from other mirrors, and so even if they did mirror things in teh right order they may be pulling from a source which did | 14:56 |
| fungi | not and was out of sync | 14:56 |
| *** che-arne has quit IRC | 14:56 | |
| *** KanagarajM has joined #openstack-infra | 14:57 | |
| *** andreaf has quit IRC | 14:58 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Save last setting of Toggle CI button https://review.openstack.org/123464 | 14:58 |
| *** andreaf has joined #openstack-infra | 14:58 | |
| *** jistr has quit IRC | 14:58 | |
| dims | fungi: clarkb: anyway for me to test the above? ^^ | 15:00 |
| *** datsun180b has joined #openstack-infra | 15:01 | |
| *** otherwiseguy has joined #openstack-infra | 15:01 | |
| *** jlibosva has quit IRC | 15:01 | |
| *** flaper87 is now known as flaper87|afk | 15:01 | |
| *** tonytan4ever has quit IRC | 15:02 | |
| clarkb | dims we can put it on review-dev first | 15:02 |
| dims | clarkb: ah cool | 15:03 |
| *** zns has quit IRC | 15:03 | |
| dims | i borrowed the get/set cookie methods from the status.js so at least those should work :) | 15:03 |
| *** dkranz has joined #openstack-infra | 15:04 | |
| *** baoli has quit IRC | 15:05 | |
| *** baoli has joined #openstack-infra | 15:05 | |
| *** IAX has quit IRC | 15:07 | |
| *** baoli has quit IRC | 15:07 | |
| *** ZZelle has quit IRC | 15:08 | |
| jeblair | mfer, clarkb, fungi, mordred: packagist has a generic hook system which we could use in the same way that we trigger rtfd builds | 15:08 |
| *** che-arne|2 has joined #openstack-infra | 15:08 | |
| *** ZZelle has joined #openstack-infra | 15:08 | |
| jeblair | mfer, clarkb, fungi, mordred: (in fact, they seem very similar -- they both build daily unless otherwise triggered by a hook) | 15:08 |
| *** nadya has joined #openstack-infra | 15:08 | |
| clarkb | oh that makes sense. a simple please go do this for me job | 15:08 |
| jeblair | mfer, clarkb, fungi, mordred: https://packagist.org/about | 15:08 |
| *** zns has joined #openstack-infra | 15:09 | |
| jeblair | unfortunately, it requires a username and api token :/ | 15:09 |
| *** nadya is now known as Guest95853 | 15:09 | |
| jeblair | which is silly | 15:09 |
| fungi | presumably to upload those json configuration blobs | 15:09 |
| mfer | jeblair openstack could have an account | 15:09 |
| *** che-arne|2 is now known as che-arne | 15:09 | |
| mfer | i was more or less asking for other cases as well. this was just one example | 15:09 |
| sdague | phschwartz: any updates? basically 100% of openstack is blocked at the moment because of that rax mirror issue | 15:10 |
| mfer | i was using other tools this am i wished i could use with openstack | 15:10 |
| jeblair | mfer: yeah, if openstack had an account and "owned" all the packagist repos, that would work fairly easily | 15:10 |
| *** amuller_ has joined #openstack-infra | 15:11 | |
| *** IAX has joined #openstack-infra | 15:11 | |
| *** mwagner_lap has quit IRC | 15:11 | |
| *** baoli has joined #openstack-infra | 15:11 | |
| jeblair | packagist naming is like github naming with a "org/" prefix; so that sort of makes sense | 15:11 |
| *** amuller__ has joined #openstack-infra | 15:11 | |
| jeblair | the fact that the name "can not change" is a little worrying | 15:12 |
| *** jergerber has joined #openstack-infra | 15:12 | |
| *** amuller has quit IRC | 15:13 | |
| sdake | morning | 15:14 |
| *** sdake has quit IRC | 15:14 | |
| *** sdake has joined #openstack-infra | 15:14 | |
| jeblair | clarkb, fungi: my local dib run did not succeed, though i do not have the logs to say why :/ | 15:14 |
| jeblair | it is trying again | 15:15 |
| clarkb | jeblair ok I think best next step is better logging | 15:15 |
| *** amuller_ has quit IRC | 15:15 | |
| jeblair | i'm sure it logged, i just didn't start this session logging to a file :( | 15:15 |
| clarkb | oh I see | 15:16 |
| jeblair | it's running puppet right now, which is near the end of the dib elements | 15:16 |
| clarkb | do you have a qcow file in your imagesdir? | 15:16 |
| *** tonytan4ever has joined #openstack-infra | 15:16 | |
| jeblair | nope | 15:17 |
| *** ihrachyshka has quit IRC | 15:17 | |
| *** markmcclain has quit IRC | 15:18 | |
| sdague | jeblair: are you the current owner of the toggle ci js ? | 15:19 |
| sdague | I was kind of curious why it's bottom up parsing the comments instead of top down | 15:20 |
| yolanda | hi, saw last week the dib stuff was uploaded, how is that working? is that progressing fine? | 15:20 |
| rcarrillocruz | i think clarkb pushed some change for dib tests | 15:21 |
| sdague | because it means it sometimes merges results | 15:21 |
| rcarrillocruz | remember seeing something on my mailbox | 15:21 |
| rcarrillocruz | yolanda: ^ | 15:21 |
| jeblair | sdague: i don't consider myself the owner. i only did that because someone else wrote toggle-ci which would have hidden all the results. :) | 15:21 |
| *** bdpayne has joined #openstack-infra | 15:21 | |
| jeblair | sdague: i suspect the originator was a drive-by. it's probably un-owned. | 15:22 |
| yolanda | rcarrillocruz, only changes for tests or some other config change? | 15:22 |
| sdague | jeblair: ok, so the reason is 'because it was that way' | 15:22 |
| sdague | not for other unknown reasons | 15:22 |
| jeblair | sdague: i wrote the comment parsing though | 15:22 |
| jeblair | sdague: i'm working on a real answer to your q | 15:22 |
| *** Longgeek has quit IRC | 15:23 | |
| rcarrillocruz | can't remember, i can look it up in a bit | 15:23 |
| rcarrillocruz | dog walk | 15:23 |
| *** homeless_ has joined #openstack-infra | 15:23 | |
| jeblair | sdague: it should bucket the results by patchset and use the most recent patchsets results | 15:23 |
| sdague | so it seems that if we instead did $(".commentPanel").each | 15:23 |
| clarkb | yolanda: we are currently trying to use it. had to write a patch to the builder to handle config changes. now our image build is apparently failing on jeblairs test machine. need logs for that | 15:24 |
| jeblair | sdague: if it's going backwards, that seems wrong, because going forward should get you the most recent run within a patchset that had >1 run | 15:24 |
| sdague | jeblair: well, it gets whacky if it passed check, failed gate | 15:24 |
| yolanda | clarkb, i'm doing tests as well, but related to debian | 15:24 |
| sdague | or if we have multiple runs on a patchset | 15:24 |
| yolanda | clarkb, can you refer me to the patch you wrote? | 15:24 |
| yolanda | i'm having failures i told you last week, about puppet and rubygems modules | 15:25 |
| sdague | because it's binning on the ps | 15:25 |
| yolanda | but that looks as a debian issue | 15:25 |
| sdague | so I was thinking a top down parsing of $(".commentPanel").each would provide a set of results objects | 15:25 |
| clarkb | yolanda https://review.openstack.org/#/c/123283/ is the patch | 15:25 |
| *** homeless has quit IRC | 15:26 | |
| sdague | anyway, I started looking mostly because what I definitely need is the timestamp of the results with it | 15:26 |
| yolanda | clarkb, ok, so to grab dinamically paths if they change | 15:27 |
| sdague | because trying to figure out if the results are the results of a recheck, or if the recheck is outstanding, is currently not possible without toggling the ci back on | 15:27 |
| clarkb | yolanda yup | 15:27 |
| yolanda | also removing all the NODEPOOL_ env vars? | 15:27 |
| jroll | sdague: mirrors are being synced | 15:27 |
| jeblair | sdague: where do you see that it's going bottom up? | 15:27 |
| sdague | it start's with looping through <p> | 15:28 |
| clarkb | yolanda that was redundant because we cloned the existing env | 15:28 |
| jeblair | sdague: yeah, that should still be ordered top-down | 15:28 |
| sdague | then goes and finds it's name via .parent().parent().parent() | 15:28 |
| *** gokrokve_ has quit IRC | 15:28 | |
| yolanda | clarkb, thx, i'll apply that to my tests also | 15:28 |
| sdague | sorry, I mean bottom up in the html structure | 15:28 |
| sdague | not in list order | 15:28 |
| jeblair | sdague: okay, what's wrong with that? | 15:29 |
| sdague | it mergers check and gate results | 15:29 |
| sdague | for the same patchset | 15:29 |
| *** IvanBerezovskiy has left #openstack-infra | 15:29 | |
| jeblair | sdague: not in that function; that function outputs a list of ci-related comments in the original order | 15:29 |
| *** jlibosva has joined #openstack-infra | 15:29 | |
| sdague | sure | 15:29 |
| jeblair | sdague: (no merging happens) | 15:29 |
| jeblair | sdague: in the ci_update_table function is where results are combined | 15:30 |
| jeblair | sdague: if a patch runs check and gate, the gate jobs should end up replacing the check jobs of the same name | 15:31 |
| jeblair | sdague: if they have different names, you'll see both | 15:31 |
| sdague | but they don't have the same names | 15:31 |
| sdague | all the time | 15:31 |
| sdague | given that we do check- and gate- tests | 15:31 |
| sdague | some times | 15:31 |
| sdague | but some times we don't | 15:31 |
| *** changbl_ has joined #openstack-infra | 15:31 | |
| sdague | basically the test results you are looking at are confusing, because they aren't actually the current test set that's failed | 15:32 |
| jeblair | right. but all the info is there; if the question is "what failed", well, the red FAILED next to "gate-foo" will tell you that.. | 15:32 |
| jeblair | they are the most recent runs of each of those jobs for the current patchset | 15:32 |
| sdague | ok | 15:32 |
| jeblair | the reason this is important, and you can't just drop any previous runs... | 15:33 |
| *** dimtruck is now known as zz_dimtruck | 15:33 | |
| jeblair | is that experimental and other things need to layer on top | 15:33 |
| jeblair | eg, if you do 'check experimental', you want to see those results as well as the check tests, not instead of | 15:33 |
| sdague | but if we keep this approach, how would you suggest putting the date of the run | 15:33 |
| *** zz_dimtruck is now known as dimtruck | 15:33 | |
| sdague | hmmm... yeh | 15:33 |
| sdague | so right now I have a ux issue that without the date of the results being returned, it's kind of hard to figure out if they are fresh | 15:34 |
| *** david-ly_ has joined #openstack-infra | 15:34 | |
| openstackgerrit | Craig Bryant proposed a change to openstack-infra/config: Add python and pypi jobs for monasca-persister https://review.openstack.org/122231 | 15:35 |
| *** jistr has joined #openstack-infra | 15:35 | |
| jeblair | sdague: yeah, we could give it a try; i'm a little worried about column width and information overload, but i don't have any immediate other ideas | 15:36 |
| *** david-lyle has quit IRC | 15:36 | |
| *** gokrokve has joined #openstack-infra | 15:37 | |
| *** gyee has joined #openstack-infra | 15:37 | |
| *** e0ne has quit IRC | 15:37 | |
| clarkb | jeblair is the image build still going? | 15:37 |
| *** funzo has joined #openstack-infra | 15:37 | |
| sdague | so I would envision adding it to the header where Jenkins is | 15:38 |
| clarkb | it was taking me about 15 minutes when I tested if I had a warm cache | 15:38 |
| sdague | <td>Jenkins</td><td>Sept 20, 15:05 </td> | 15:38 |
| jeblair | sdague: ah that would be nice, but because of the combining, the individual jobs would have different times | 15:39 |
| sdague | right | 15:39 |
| jeblair | sdague: we could gloss over that and just put the latest of any jenkins run | 15:39 |
| *** bhuvan has joined #openstack-infra | 15:39 | |
| sdague | yeh, that might be good enough for now | 15:39 |
| sdague | we can fix the other later | 15:39 |
| jeblair | sdague: or could try to have it detect runs -- eg, this run had the same jobs as this other run, therefore replace it, but otherwise keep them in separate bins (so you'd have a jenkins section for check/gate/experimental) | 15:39 |
| *** mfer has quit IRC | 15:39 | |
| jeblair | that's, erm, more complicated :) | 15:39 |
| sdague | sure, but that was my thinking about a top down parsing | 15:40 |
| sdague | if we build a set of objects that are results | 15:40 |
| sdague | we can manip them later | 15:40 |
| jeblair | that's what ci_find_comments does | 15:40 |
| sdague | except it merges in the systems[] right? | 15:40 |
| jeblair | no, that's ci_update_table | 15:40 |
| jeblair | ci_find_comments does nothing other than extract the data with zero manipulation | 15:41 |
| jroll | sdague: hey, any idea what region these failures were in? | 15:42 |
| jeblair | ci_update_table is the method to change to bucket results by run | 15:42 |
| sdague | jeblair: ok, well in the current structure my brain isn't really groking it | 15:42 |
| jroll | sdague: (the mirror failures, that is) | 15:42 |
| *** dannywilson has joined #openstack-infra | 15:43 | |
| jeblair | sdague: ci_find_comments produces an ordered list of data structures for every comment by a ci system. the data structure it produces for each comment is here: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/gerrit/hideci.js#n50 | 15:44 |
| jeblair | that's it. no manipulation of the data, just a list of comments. | 15:44 |
| jeblair | sdague: the heavily commented ci_update_table tries to make sense of those and display only the results we're interested in. | 15:45 |
| jeblair | (it builds hashes of hashes of hashes, and gets kinda complicated) | 15:46 |
| *** toni_ has joined #openstack-infra | 15:46 | |
| *** mwagner_lap has joined #openstack-infra | 15:46 | |
| toni_ | I can not install devstack with icehouse I got the error: 'amqplib' is not in global-requirements.txt | 15:46 |
| jeblair | clarkb, fungi: finished; it looks like my local test host may have run out of space. maybe we give it a go for real? | 15:48 |
| clarkb | jeblair I am game for it. need to get to a spot where I can restart nodepool though | 15:49 |
| fungi | jeblair: sure, unless you think you can free more room and want to try again | 15:49 |
| clarkb | note nodepool.o.o is old style server with giant / so should be happy | 15:49 |
| fungi | but per lifeless's comment, we may want to symlink ~/.cache into somewhere we have better control over | 15:50 |
| jeblair | fungi: i will, but i think this process takes a veeeery long time | 15:50 |
| clarkb | toni_ you can usually get better luck debugging devstack in #openstack-qa | 15:50 |
| jeblair | fungi: my .cache ended up being 3.6G | 15:50 |
| fungi | yeah, that's pretty sizable | 15:50 |
| clarkb | that sounds right. base image + packages + devstack vmimages + git repos | 15:51 |
| *** gokrokve has quit IRC | 15:51 | |
| *** pballand has joined #openstack-infra | 15:52 | |
| *** jgrimm is now known as zz_jgrimm | 15:53 | |
| *** hashar has joined #openstack-infra | 15:53 | |
| *** dannywilson has quit IRC | 15:55 | |
| *** dannywilson has joined #openstack-infra | 15:55 | |
| *** baoli has quit IRC | 15:56 | |
| zaro | morning | 15:56 |
| anteaya | morning zaro | 15:56 |
| ekarlso | phschwartz: yo mr, you up ? | 15:57 |
| *** VijayTripathi has joined #openstack-infra | 15:59 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 15:59 | |
| *** dannywilson has quit IRC | 15:59 | |
| *** dannywilson has joined #openstack-infra | 15:59 | |
| *** mfer has joined #openstack-infra | 16:00 | |
| *** arxcruz has quit IRC | 16:00 | |
| clarkb | hrm doesnt look like we are getting enough test nodes. is now a good time to restart nodepool? | 16:00 |
| clarkb | jeblair fungi ^ | 16:00 |
| fungi | clarkb: might be a good time for me to check what's up at least | 16:01 |
| fungi | might have leaked floating ips again... checking | 16:01 |
| *** hashar is now known as hasharConfcall | 16:02 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Doc updates for project-config https://review.openstack.org/122908 | 16:03 |
| *** james_li has joined #openstack-infra | 16:04 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Doc updates for project-config https://review.openstack.org/122908 | 16:04 |
| *** Guest95853 has quit IRC | 16:04 | |
| *** toni_ has quit IRC | 16:04 | |
| fungi | no fip cruft in hpcloud at least | 16:05 |
| fungi | rax-dfw is struggling a bit on launch failures compared to other providers/regions (by an order of magnitude or more) but not an outrageously high count | 16:06 |
| *** liusheng has quit IRC | 16:09 | |
| *** rmcall has joined #openstack-infra | 16:09 | |
| *** liusheng has joined #openstack-infra | 16:10 | |
| *** baoli has joined #openstack-infra | 16:11 | |
| fungi | i suspect it's just the constant gate resets churning our pool harder than usual | 16:11 |
| clarkb | that makes sense | 16:12 |
| clarkb | give me a couple minutes and I will be ready to restart nodepool | 16:12 |
| fungi | yeah, we didn't have a bunch of nodes stuck in any particular state for longer than i would consider typical | 16:13 |
| fungi | other than the usual handful of undeletables | 16:13 |
| *** nadya_ has joined #openstack-infra | 16:13 | |
| jeblair | fungi, clarkb: don't restart nodepool yet; i want to test something | 16:14 |
| fungi | k | 16:14 |
| *** baoli has quit IRC | 16:15 | |
| clarkb | jeblair: rgr | 16:15 |
| *** marcoemorais has joined #openstack-infra | 16:16 | |
| *** baoli has joined #openstack-infra | 16:16 | |
| *** baoli has quit IRC | 16:16 | |
| *** baoli has joined #openstack-infra | 16:19 | |
| clarkb | morganfainberg: the postgres-full job runtime average is a lot closer to the other job now. 53.89 minutes. Pretty sure your change fixed it | 16:19 |
| morganfainberg | clarkb, yay! | 16:19 |
| morganfainberg | clarkb, yeah uuid tokens + 1 eventlet worker = sad gate | 16:19 |
| *** markmcclain has joined #openstack-infra | 16:20 | |
| *** markmcclain has quit IRC | 16:21 | |
| *** BobBall is now known as BobBall_Away | 16:22 | |
| *** markmcclain has joined #openstack-infra | 16:22 | |
| *** tonytan4ever has quit IRC | 16:22 | |
| *** tonytan4ever has joined #openstack-infra | 16:23 | |
| *** dane_leblanc has quit IRC | 16:26 | |
| *** tonytan4ever has quit IRC | 16:27 | |
| *** Guest34961 has quit IRC | 16:28 | |
| jeblair | clarkb, fungi: i believe dib expects sudo access | 16:28 |
| clarkb | jeblair: oh right it does | 16:29 |
| fungi | oho... it uses chroot, right? | 16:29 |
| clarkb | gah :/ | 16:29 |
| *** dannywilson has quit IRC | 16:29 | |
| clarkb | fungi: yup | 16:29 |
| fungi | can't chroot if you're not root | 16:29 |
| * fungi likes catchy rhyming phrases | 16:29 | |
| *** dannywilson has joined #openstack-infra | 16:29 | |
| *** hdd has joined #openstack-infra | 16:29 | |
| jeblair | considering nodepool's lack of network daemon, i think it's probably okay to give it passwordless sudo | 16:31 |
| *** larsks has left #openstack-infra | 16:31 | |
| *** rushiagr is now known as rushiagr_away | 16:31 | |
| jeblair | though we should probably split this up and make dib gearman workers for it and only give them access in the long run | 16:31 |
| *** rushiagr_away is now known as rushiagr | 16:32 | |
| morganfainberg | mordred, i know you had some thoughts on keystoneclient etc, if you have anything specific to add to the mix feel free to add it here: https://etherpad.openstack.org/p/keystone-kilo-summit-sessions (our discussion on potential summit sessions) | 16:32 |
| *** dannywilson has quit IRC | 16:32 | |
| *** dannywilson has joined #openstack-infra | 16:32 | |
| clarkb | jeblair: ++ | 16:32 |
| fungi | yeah, unfortunately even if we restrict sudo just to the chroot command, root in a chroot can easily escape to global root context so it's not really an effective security measure | 16:33 |
| clarkb | everything nodepool does is as a client to other things. it seems relatively safe to run like that | 16:33 |
| jeblair | clarkb: do you want to whip up a change to add it? | 16:33 |
| clarkb | fungi: I think it also needs mount, and other things too | 16:33 |
| clarkb | jeblair: sure | 16:33 |
| *** amuller has joined #openstack-infra | 16:34 | |
| markvan_ | clarkb fungi: when ya get a chance..... thx https://review.openstack.org/#/c/121958/3 | 16:34 |
| *** vdo has quit IRC | 16:35 | |
| *** amuller__ has quit IRC | 16:35 | |
| *** rushiagr is now known as rushiagr_away | 16:36 | |
| *** daya_k has quit IRC | 16:36 | |
| zaro | clarkb: would you mind taking a look? https://review.openstack.org/#/c/116347 | 16:37 |
| *** jp_at_hp has quit IRC | 16:37 | |
| *** jgallard has quit IRC | 16:38 | |
| *** larsks has joined #openstack-infra | 16:38 | |
| SergeyLukjanov | AJaeger, I think anyone can do it | 16:39 |
| *** afazekas has quit IRC | 16:40 | |
| SergeyLukjanov | AJaeger, I'm expecting CR author doing it | 16:40 |
| anteaya | SergeyLukjanov: are you here or not here? | 16:40 |
| SergeyLukjanov | anteaya, I'm here | 16:40 |
| SergeyLukjanov | here == in California till mid October | 16:41 |
| SergeyLukjanov | but I'm mostly occupied on the mirantis internal stuff and sahara release | 16:41 |
| anteaya | SergeyLukjanov: okay do you have time to process the outstanding gerrit ci account requests? | 16:41 |
| anteaya | SergeyLukjanov: ah, clarkb said he would if you don't have time | 16:41 |
| SergeyLukjanov | if you have something specific in infra to do - ping me please | 16:41 |
| anteaya | outstanding ci account requests should be on the third-party-requests list | 16:42 |
| SergeyLukjanov | anteaya, nono, I can do it, just no always reading channel logs if there's no highlights on my name ;) | 16:42 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Give nodepool user sudo access for dib https://review.openstack.org/123507 | 16:42 |
| clarkb | jeblair: fungi ^ | 16:42 |
| anteaya | SergeyLukjanov: right, well all the details should be in the mailing list threads | 16:42 |
| *** SumitNaiksatam has quit IRC | 16:42 | |
| anteaya | SergeyLukjanov: let me know if you think something is missing or if you have a question | 16:42 |
| anteaya | SergeyLukjanov: and thanks | 16:43 |
| SergeyLukjanov | anteaya, thx, I'll go through the requests later today | 16:43 |
| openstackgerrit | A change was merged to openstack/requirements: Add a tox env to check requirements overlap https://review.openstack.org/98262 | 16:43 |
| *** zz_jgrimm is now known as jgrimm | 16:43 | |
| anteaya | and I haven't been pinging you because you need your time to be, y'know, married | 16:43 |
| *** patrickeast has joined #openstack-infra | 16:43 | |
| anteaya | SergeyLukjanov: thanks | 16:43 |
| *** shashankhegde has joined #openstack-infra | 16:43 | |
| *** koolhead17 has joined #openstack-infra | 16:43 | |
| clarkb | zaro: I rechecked that change so that we can see the docs-draft output | 16:45 |
| clarkb | zaro: looks like the tests ran long enough ago that the docs draft build was already deleted | 16:45 |
| *** bhuvan has quit IRC | 16:46 | |
| *** HeOS has quit IRC | 16:46 | |
| *** gokrokve has joined #openstack-infra | 16:48 | |
| *** yfried is now known as yfried|afk | 16:52 | |
| *** todd_dsm has joined #openstack-infra | 16:52 | |
| *** e0ne has joined #openstack-infra | 16:53 | |
| *** shashankhegde has quit IRC | 16:54 | |
| *** ihrachyshka has joined #openstack-infra | 16:54 | |
| *** dteselkin_ is now known as dteselkin | 16:57 | |
| *** hasharConfcall has quit IRC | 16:57 | |
| *** koolhead17 has quit IRC | 17:01 | |
| *** larsks has left #openstack-infra | 17:01 | |
| *** koolhead17 has joined #openstack-infra | 17:01 | |
| *** derekh has quit IRC | 17:03 | |
| *** loki184 has joined #openstack-infra | 17:03 | |
| openstackgerrit | A change was merged to openstack/requirements: add integration test script from pbr https://review.openstack.org/118410 | 17:04 |
| *** bhuvan has joined #openstack-infra | 17:04 | |
| *** david-ly_ is now known as david-lyle | 17:04 | |
| *** stevemar has quit IRC | 17:04 | |
| *** stevemar has joined #openstack-infra | 17:05 | |
| *** zns has quit IRC | 17:06 | |
| *** loki184 has quit IRC | 17:08 | |
| *** rushiagr_away is now known as rushiagr | 17:08 | |
| anteaya | stevemar: let me know when you bring this one out of hibernation in case it falls off my radar: https://review.openstack.org/#/c/122766/ | 17:08 |
| *** bhuvan has quit IRC | 17:09 | |
| *** bhuvan has joined #openstack-infra | 17:09 | |
| anteaya | if anyone has the time, can you tell me what this means? https://review.openstack.org/#/c/122922/2/modules/openstack_project/files/review.projects.yaml I don't understand the reply | 17:09 |
| anteaya | perhaps this is due to not understanding heat | 17:10 |
| *** kgiusti has quit IRC | 17:10 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:10 | |
| *** praneshp has joined #openstack-infra | 17:11 | |
| clarkb | your interesting semi useful information of the day: the US will roll clocks back Nov 2nd to end DST. Paris will roll back a week earlier. So if you fly between october 26 and november 2 from the US or canada you will essentially miss the switch \o/ | 17:11 |
| clarkb | anteaya: heat is an orchestration engine so you can have it do things automatically for you | 17:12 |
| *** kgiusti has joined #openstack-infra | 17:12 | |
| clarkb | anteaya: it sounds like they itnend on using this featuer to do stuf faround group policy abstractions (note I have no idea what that means) | 17:12 |
| anteaya | clarkb: yay for missing the time change | 17:12 |
| jeblair | clarkb: that describes me! yay! | 17:13 |
| *** e0ne is now known as e0ne_ | 17:13 | |
| *** baoli has quit IRC | 17:13 | |
| jeblair | of course, this also does not affect fungi ;) | 17:13 |
| fungi | of course | 17:13 |
| anteaya | clarkb: yeah, I understand all the words, I just have zero idea what it means | 17:14 |
| clarkb | he operates on utc so he should be fine | 17:14 |
| jeblair | fungi: but at least you don't have to wonder why all the stores suddenly changed their hours :) | 17:14 |
| fungi | this is true | 17:14 |
| *** AJaeger_ has joined #openstack-infra | 17:14 | |
| anteaya | clarkb: I was hopeing someone else whould know what it means | 17:15 |
| anteaya | sounds like no | 17:15 |
| anteaya | sigh | 17:15 |
| clarkb | jeblair: we get to experience jetlag instead of dst change. somehow I think this is better | 17:15 |
| *** loki184 has joined #openstack-infra | 17:15 | |
| JayF | clarkb: I noticed that earlier; and it is awesome :D </3 DST | 17:17 |
| AJaeger_ | SergeyLukjanov, anteaya : I've registered python-manilaclient now - could you review https://review.openstack.org/#/c/122946/ again, please? | 17:17 |
| * anteaya reviews | 17:17 | |
| *** rwsu has quit IRC | 17:17 | |
| stevemar | anteaya, oh don't worry, i won't forget about it :) | 17:18 |
| *** daya_k has joined #openstack-infra | 17:18 | |
| anteaya | AJaeger_: thanks | 17:18 |
| anteaya | stevemar: yay | 17:18 |
| anteaya | enjoy working with people I can count on | 17:19 |
| openstackgerrit | Ramy Asselin proposed a change to openstack-infra/devstack-gate: Add a pre-clean hook. https://review.openstack.org/122896 | 17:19 |
| *** harlowja_away is now known as harlowja | 17:19 | |
| * anteaya tries to get a walk in before meeting season | 17:20 | |
| *** pblaho has quit IRC | 17:20 | |
| fungi | heh... meeting season | 17:21 |
| fungi | it is like giant-meeting-block | 17:21 |
| fungi | kaiju meeting | 17:22 |
| *** torandu has joined #openstack-infra | 17:22 | |
| *** vhoward has joined #openstack-infra | 17:23 | |
| *** amcrn has joined #openstack-infra | 17:23 | |
| *** shashankhegde has joined #openstack-infra | 17:24 | |
| *** che-arne has quit IRC | 17:25 | |
| *** e0ne_ has quit IRC | 17:25 | |
| *** praneshp has quit IRC | 17:25 | |
| clarkb | jeblair: were you able to rerun on your side with sudo allowed? | 17:25 |
| clarkb | jeblair: I suppose disk space will still be an issue | 17:26 |
| anteaya | fungi: :D | 17:26 |
| *** baoli has joined #openstack-infra | 17:26 | |
| *** tonytan4ever has joined #openstack-infra | 17:27 | |
| *** markmcclain has quit IRC | 17:28 | |
| *** praneshp has joined #openstack-infra | 17:29 | |
| *** rdopieralski has quit IRC | 17:30 | |
| *** killerintersekt has joined #openstack-infra | 17:31 | |
| *** stevemar has quit IRC | 17:31 | |
| *** dannywilson has quit IRC | 17:31 | |
| *** stevemar has joined #openstack-infra | 17:32 | |
| *** dannywilson has joined #openstack-infra | 17:32 | |
| *** e0ne has joined #openstack-infra | 17:35 | |
| *** rwsu has joined #openstack-infra | 17:36 | |
| *** maxbit has joined #openstack-infra | 17:36 | |
| *** emagana has joined #openstack-infra | 17:36 | |
| *** dannywilson has quit IRC | 17:36 | |
| *** yfried|afk is now known as yfried | 17:37 | |
| clarkb | jeblair: fungi: I am doing more "performance" testing of the image builds and on my hp test node, it takes about half an hour to do a devstack iamge with no cache. With a warm cache it takes just under 15 minutes. Most of the time is spent in the image conversion step. This step appears cpu bound. | 17:38 |
| jeblair | clarkb: with sudo it got pretty much to the end; tbh i'm not entirely sure what the error at the end was, though i discovered perms were wrong on the output dir. i'm running again in case that was the problem | 17:38 |
| jeblair | clarkb: what size node do you have? | 17:38 |
| clarkb | jeblair: 30gigawatt | 17:38 |
| clarkb | because that has a bigger / | 17:38 |
| jeblair | ah yeah, i'm only using a 1.0jiggawatt node | 17:38 |
| *** killerintersekt has quit IRC | 17:39 | |
| yolanda | clarkb, so dib finally worked? where were you having issues? | 17:39 |
| clarkb | yolanda: I am not using it with nodepool like jeblair is | 17:39 |
| jeblair | it is much slower. but i formatted the ephemeral disk, so i think i have enough space | 17:39 |
| clarkb | yolanda: I am just running it directly. To minimize the variables int he image build process. Image builds directly work fine | 17:39 |
| clarkb | yolanda: and I think we are really close to having it work with nodepool | 17:39 |
| *** JayJ has joined #openstack-infra | 17:40 | |
| clarkb | yolanda: there is a change going through the gate that will allow the nodepool user to sudo because dib needs that | 17:40 |
| clarkb | yolanda: once that is in we need to restart nodepool and give it a go (assuming jeblair's tests indicate success) | 17:40 |
| yolanda | clarkb, do you have the changes tagged with some keyword? | 17:40 |
| jeblair | in the mean time, i'm hacking on logging | 17:40 |
| clarkb | yolanda: I don't sorry | 17:40 |
| clarkb | jeblair: ++ | 17:40 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Clean up bashate failures https://review.openstack.org/118944 | 17:41 |
| yolanda | jeblair, that's an issue yes, when dib-image fails the log wasn't stored, right? | 17:41 |
| jeblair | yolanda: it is, but it's all in one giant entry in the main log | 17:42 |
| yolanda | i'm still fighting with debian deps but takes ages for me to do the testing, i'll be filing some patches once i have them | 17:42 |
| jeblair | i'm working on having it do line-by-line to the image log | 17:42 |
| yolanda | cool, that would be nice | 17:42 |
| *** atiwari has joined #openstack-infra | 17:42 | |
| yolanda | i have to add debootstrap package as well to the debian dib element, i'll be filing something | 17:43 |
| *** loki184 has quit IRC | 17:43 | |
| *** killerintersekt has joined #openstack-infra | 17:43 | |
| *** dannywilson has joined #openstack-infra | 17:43 | |
| yolanda | jeblair, there's also that patch pending https://review.openstack.org/#/c/104949/ | 17:44 |
| yolanda | if not, that wasn't working on HP Cloud | 17:44 |
| AJaeger_ | clarkb, jeblair, fungi : I rebase 118944 and suggest to add it so that bashate passes the config tests. Could you review it, please? | 17:44 |
| jeblair | yolanda: so what we're doing won't work until that change lands? | 17:45 |
| yolanda | at least it was failing for me when i did the initial ubuntu testing on hp cloud | 17:45 |
| *** VijayTripathi has quit IRC | 17:46 | |
| yolanda | honestly i haven't tried in Ubuntu for some time | 17:46 |
| clarkb | I think we want cloud init | 17:46 |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/nodepool: Reset the client object before creating the image https://review.openstack.org/74943 | 17:46 |
| clarkb | dont hp images cloud init by default? | 17:46 |
| jeblair | clarkb: i'm pretty sure in the long run that we don't. :) | 17:46 |
| *** rushiagr is now known as rushiagr_away | 17:46 | |
| *** VijayTripathi has joined #openstack-infra | 17:46 | |
| *** mriedem is now known as mriedem_afk | 17:46 | |
| yolanda | clarkb, jeblair, in hp cloud, i was receiving some errors because DataSource None didn't exist, from what i remember | 17:47 |
| jeblair | clarkb: i can't imagine what we would want it for | 17:47 |
| *** killerintersekt has quit IRC | 17:49 | |
| *** bhuvan has quit IRC | 17:50 | |
| clarkb | ya we probably dont need it | 17:50 |
| yolanda | so i got a -1 for that, because then it needs updates on tests of other projects | 17:52 |
| mordred | we do not want cloud init | 17:53 |
| mordred | actively | 17:53 |
| mordred | it serves no purpose for us | 17:53 |
| jeblair | yolanda: what do i set to "fake-dib-image" to use the fake code path in dib? | 17:53 |
| jeblair | (in nodepool) | 17:53 |
| *** dprince has quit IRC | 17:54 | |
| *** pcrews has joined #openstack-infra | 17:54 | |
| yolanda | jeblair, not sure to follow you | 17:54 |
| *** VijayTripathi has quit IRC | 17:54 | |
| jeblair | yolanda: grep for "fake-dib-image" in nodepool; how do i use that? | 17:54 |
| jeblair | diskimages: - name: fake-dib-image | 17:55 |
| jeblair | yolanda: do i do that ^? | 17:55 |
| yolanda | jeblair | 17:55 |
| yolanda | labels: | 17:55 |
| yolanda | - name: fake-dib-label | 17:55 |
| yolanda | image: fake-dib-image | 17:55 |
| krotscheck | anteaya: I’m not certain the first item on the infra agenda is still necessary, is there something you wanted to talk about re: project group/groups? | 17:55 |
| krotscheck | Sorry, not first item. The group/groups item. | 17:55 |
| jeblair | yolanda: cool, thanks | 17:56 |
| *** bhuvan has joined #openstack-infra | 17:56 | |
| yolanda | diskimages: | 17:56 |
| yolanda | - name: fake-dib-image | 17:56 |
| yolanda | elements: | 17:56 |
| yolanda | - ubuntu | 17:56 |
| yolanda | - vm | 17:56 |
| *** jistr has quit IRC | 17:57 | |
| *** rushiagr_away is now known as rushiagr | 17:57 | |
| clarkb | yolanda that error seems familiar. was it a cloud init bug? | 18:00 |
| *** yfried is now known as yfried|afk | 18:00 | |
| yolanda | clarkb, i remember that DataSource None was hardcoded on the cloud init element, and HP Cloud didn't have it supported | 18:01 |
| *** rmcall has quit IRC | 18:01 | |
| *** rmcall has joined #openstack-infra | 18:02 | |
| *** unicell has quit IRC | 18:03 | |
| *** jlibosva has quit IRC | 18:06 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: updated requirements repo readme https://review.openstack.org/116708 | 18:06 |
| *** homeless_ has quit IRC | 18:06 | |
| *** maxbit has quit IRC | 18:06 | |
| *** amotoki has quit IRC | 18:07 | |
| *** esker has joined #openstack-infra | 18:07 | |
| *** maxbit has joined #openstack-infra | 18:07 | |
| *** homeless has joined #openstack-infra | 18:07 | |
| *** maxbit_ has joined #openstack-infra | 18:10 | |
| *** ChuckC_ has quit IRC | 18:10 | |
| *** nadya_ has quit IRC | 18:10 | |
| *** wenlock has joined #openstack-infra | 18:11 | |
| *** maxbit has quit IRC | 18:11 | |
| *** otherwiseguy has quit IRC | 18:12 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Line-by-line dib logs https://review.openstack.org/123528 | 18:12 |
| jeblair | yolanda, clarkb: ^ | 18:12 |
| *** james_li has left #openstack-infra | 18:12 | |
| *** VijayTripathi has joined #openstack-infra | 18:12 | |
| jeblair | i've tested that with the fake system but not for real | 18:13 |
| jeblair | (waiting for my most recent real image to finish building) | 18:13 |
| *** esker has quit IRC | 18:13 | |
| *** cnesa15 has quit IRC | 18:14 | |
| *** dizquierdo has quit IRC | 18:14 | |
| *** ociuhandu has quit IRC | 18:15 | |
| *** nadya_ has joined #openstack-infra | 18:15 | |
| fungi | jeblair: clarkb: to tackle bug 1360324 it appears i need accessbot to grant me +s on the channels i'm forwarding between... should i temporarily move my nick from operators to masters in channels.yaml, or add +s to the list for operators, or create a new access level for this purpose, or...? | 18:16 |
| uvirtbot | Launchpad bug 1360324 in openstack-ci "Move #openstack-packaging IRC channel to #openstack-stable" [Medium,In progress] https://launchpad.net/bugs/1360324 | 18:16 |
| *** esker has joined #openstack-infra | 18:18 | |
| *** arnaud has joined #openstack-infra | 18:18 | |
| anteaya | krotscheck: no you explained that storyboard groups are deletable, which addresses my concern, I'm good, thanks for checking | 18:19 |
| jeblair | fungi: how exciting! | 18:19 |
| jeblair | i've wondered how to do this | 18:19 |
| *** intersekted has joined #openstack-infra | 18:19 | |
| fungi | yeah, you end up setting "forward on uninvited" on the old channel and then making it guarded invite-only and mode-locking it | 18:20 |
| jeblair | aha! | 18:20 |
| *** sarob is now known as sarob_away | 18:20 | |
| *** dprince has joined #openstack-infra | 18:21 | |
| sdague | fungi: so we've got a ton of deleting nodes again on zuul. Is there a flush to be done? | 18:21 |
| *** praneshp has quit IRC | 18:21 | |
| fungi | it's neat, but a little vague from the chanserv help/freenode docs. i had to mostly figure out the process myself before i had the right terminology to be able to pick suitable keywords to locate that ubuntu community wiki article | 18:21 |
| jeblair | fungi: i think it would be okay to add s to operators | 18:22 |
| fungi | sdague: when i looked, it seemed we were just overloading providers with delete requests and many were being ignored and falling into the periodic retry queue | 18:22 |
| fungi | sdague: i think the rapid churn from the mirror issues earlier was taking its toll, causing us to request boot and delete more rapidly than usual | 18:23 |
| fungi | i'll check again though | 18:23 |
| sdague | ok | 18:23 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Updated from global requirements https://review.openstack.org/123311 | 18:24 |
| openstackgerrit | A change was merged to openstack-infra/config: Give nodepool user sudo access for dib https://review.openstack.org/123507 | 18:24 |
| *** yfried|afk is now known as yfried | 18:25 | |
| *** Azher has joined #openstack-infra | 18:25 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add channel set permission for IRC operators https://review.openstack.org/123532 | 18:26 |
| fungi | jeblair: so basically that ^ should do it, sounds like | 18:26 |
| *** intersekted has quit IRC | 18:27 | |
| jeblair | fungi: yep | 18:28 |
| acfleury | jeblair, hi! i was running an old version of nodepool before adding the 'label' primitive. I upgraded it to current version (using puppet ci), but the database schema remains the old one. | 18:28 |
| acfleury | Do you know how can i upgdrade it? | 18:28 |
| *** ihrachyshka has quit IRC | 18:28 | |
| krotscheck | anteaya: Awesome, I’ll pull it from the agenda. Can you poke at https://review.openstack.org/#/c/111814/ when you get a chance (It’s the second of the two affiliated reviews) | 18:29 |
| fungi | acfleury: we deleted all nodes/images and dropped our database when we upgraded, i believe | 18:29 |
| jeblair | acfleury: yeah, nodepool doesn't have any schema migrations :( i think we just dropped the whole database and let it recreate it | 18:29 |
| acfleury | jeblair, fungi : i also need to recreate the images in that case? | 18:29 |
| acfleury | nodepool image-update | 18:30 |
| *** rushiagr is now known as rushiagr_away | 18:30 | |
| acfleury | or do they remain there | 18:30 |
| fungi | acfleury: you could probably manually reinsert all the details for each image into the new database, but i wouldn't begin to try to walk you through that | 18:30 |
| *** bswartz has quit IRC | 18:31 | |
| fungi | acfleury: since we update our images daily anyway, we were in pretty good shape to just regenerate them all after the upgrade | 18:31 |
| acfleury | fungi, no, i dont want to go that path. better to rebuild them if needed. | 18:31 |
| acfleury | ok. thanks fungi! | 18:31 |
| *** daya_k has quit IRC | 18:32 | |
| *** arnaud has quit IRC | 18:32 | |
| anteaya | krotscheck: +1, if you can offer the patch removing the mention of launchpad, I can review that as well | 18:32 |
| *** arnaud has joined #openstack-infra | 18:33 | |
| jeblair | clarkb, fungi: want to try to get 123528 in before nodepool restart? | 18:33 |
| clarkb | jeblair: looking at it now | 18:33 |
| clarkb | sorry was afk finishing a few things before meetings | 18:33 |
| jeblair | i'm running that for real now, and it seems to be working. it has not completed yet though. | 18:33 |
| *** praneshp has joined #openstack-infra | 18:34 | |
| fungi | jeblair: yeah, we should | 18:34 |
| jeblair | my last image build ran out of space again. i'm trying again, but at this point i think it's probably worth trying it in production (either before or after my change) | 18:34 |
| *** baoli has quit IRC | 18:34 | |
| *** HeOS has joined #openstack-infra | 18:35 | |
| *** praneshp has quit IRC | 18:35 | |
| krotscheck | anteaya: You already +1’d that one :) | 18:35 |
| anteaya | krotscheck: I'm so efficient | 18:35 |
| jeblair | i like all the smart caching dib does | 18:35 |
| jeblair | 2014-09-23 18:35:45,152 INFO nodepool.image.build.devstack-trusty-dib: Server copy has not changed. Using locally cached http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-root.tar.gz | 18:35 |
| fungi | 123528 lgtm | 18:36 |
| *** markmcclain has joined #openstack-infra | 18:37 | |
| clarkb | jeblair: ya, that was one of the big bonuses to using it | 18:37 |
| *** markmcclain has quit IRC | 18:37 | |
| clarkb | I have approved 123528 | 18:38 |
| *** praneshp has joined #openstack-infra | 18:38 | |
| lifeless | <3 | 18:38 |
| *** amuller has quit IRC | 18:38 | |
| clarkb | not only do we build each image once, but we also get cached artifacts with smart caching | 18:39 |
| *** maxbit_ has quit IRC | 18:40 | |
| *** bswartz has joined #openstack-infra | 18:41 | |
| *** lcheng has quit IRC | 18:41 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: updated requirements repo readme https://review.openstack.org/116708 | 18:42 |
| *** markmcclain has joined #openstack-infra | 18:44 | |
| *** dimtruck is now known as zz_dimtruck | 18:45 | |
| *** mriedem_afk is now known as mriedem | 18:45 | |
| *** zz_dimtruck is now known as dimtruck | 18:49 | |
| *** dane_leblanc has joined #openstack-infra | 18:49 | |
| *** zns has joined #openstack-infra | 18:50 | |
| *** wenlock has quit IRC | 18:50 | |
| mordred | jeblair: ++ | 18:50 |
| *** praneshp has quit IRC | 18:50 | |
| arosen | hrm, should my nodepool nodes be using:f http://mirror.rackspace.com/ubuntu/dists/trusty/Release.gpg ? | 18:52 |
| arosen | I don't see the string rackspace in /etc anywhere. | 18:52 |
| clarkb | arosen: if you are building on rackspace then yes. they provide mirrors and set them up in their base images | 18:53 |
| clarkb | arosen: looking /etc/apt/sources.list | 18:53 |
| clarkb | * look in | 18:53 |
| arosen | clarkb: this is on the nodepool/zuul node that i setup myself running in our cloud. | 18:53 |
| clarkb | arosen: if your base image comes from there then I would expect it. I am pretty sure our puppet does not override the default sources.list | 18:54 |
| *** ociuhandu has joined #openstack-infra | 18:54 | |
| jhesketh | Morning | 18:55 |
| arosen | clarkb: it's because of https://github.com/openstack-infra/devstack-gate/blob/master/functions.sh#L78 | 18:55 |
| anteaya | morning jhesketh | 18:55 |
| arosen | clarkb: it think we should probably change these to google.com instead. What do you think? | 18:56 |
| devananda | hi folks! I might have screwed up last week by not proposing a bump to global requirements' version of python-ironicclient -- but i'm not sure, and would like some advice from folks more knowledgable in packaging than I | 18:56 |
| clarkb | arosen: oh so they aren't using it in sources.list just hitting it. I think we use those values as a sanity check for both sides | 18:56 |
| arosen | clarkb: fwiw it seems like rax is back up now though . | 18:56 |
| jeblair | 2014-09-23 18:31:51,010 INFO nodepool.image.build.devstack-trusty-dib: umount2: Invalid argument | 18:57 |
| jeblair | 2014-09-23 18:31:51,093 INFO nodepool.image.build.devstack-trusty-dib: umount: /tmp/image.lyVl2cy4: not mounted | 18:57 |
| jeblair | that's how my local run ends. i don't know what's happening. :( | 18:57 |
| clarkb | jeblair: hrm that doesn't look familiar | 18:57 |
| clarkb | devananda: is the new version required to make new functionality work in the integrated release? | 18:57 |
| *** SumitNaiksatam has quit IRC | 18:58 | |
| devananda | clarkb: I believe the nova driver relies on it | 18:58 |
| devananda | clarkb: however, python-ironicclient isn't in nova's requirements.txt (it's in the test-requirements) | 18:59 |
| jeblair | clarkb: also, dib does a lot of work in /tmp; so i think that will need to be sizable in the future too; it would be nice if it could do its work in /opt | 18:59 |
| clarkb | devananda: does the nova driver require it? | 18:59 |
| *** esker has quit IRC | 18:59 | |
| clarkb | devananda: in that case I think you are probably ok with the way it is | 18:59 |
| *** tonytan4ever has quit IRC | 18:59 | |
| jeblair | (shouldn't be a problem on our current production host though) | 18:59 |
| *** praneshp has joined #openstack-infra | 18:59 | |
| clarkb | jeblair: ok | 18:59 |
| clarkb | ya a dib root option would be nice | 18:59 |
| fungi | meetin' time! | 19:00 |
| *** Liana_Barrows has joined #openstack-infra | 19:00 | |
| devananda | clarkb: yes, the nova driver requires it, but this is done by a runtime import in the driver, and not represented in requirements.txt | 19:00 |
| *** derekh has joined #openstack-infra | 19:00 | |
| lifeless | jeblair: dib honours the stndard tmp dir variables | 19:00 |
| clarkb | devananda: right so less important. also does it require the new version or can it work with the older one | 19:00 |
| lifeless | jeblair: just export the one you want | 19:00 |
| clarkb | devananda: it requires teh client, that is understood. but is the latest version necessary to make it work? | 19:00 |
| *** andreykurilin_ has joined #openstack-infra | 19:00 | |
| devananda | clarkb: if packagers take the current release of python-ironicclient (0.3.0) then things should be fine. if they take the one listed in g-r (0.2.1) then I believe the nova driver will error | 19:00 |
| lifeless | (to work in /opt) - if that doesn't work, its a bug | 19:00 |
| jeblair | lifeless: ack | 19:00 |
| *** hashar has joined #openstack-infra | 19:01 | |
| devananda | clarkb: that's where I'm hazy on the details | 19:01 |
| lifeless | we could also look at a cli option | 19:01 |
| clarkb | devananda: in that case you may want to consider bumping it. its basically a FFE request to do it. you probably want to not be hazy on details :) | 19:01 |
| *** praneshp has quit IRC | 19:01 | |
| devananda | clarkb: indeed | 19:01 |
| *** Liana_Barrows has quit IRC | 19:01 | |
| devananda | clarkb: i mean, hazy on what packagers will do, not what the driver will do. I'm double checking there right now | 19:02 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 19:02 | |
| *** esker has joined #openstack-infra | 19:03 | |
| *** sabeen1 has quit IRC | 19:04 | |
| *** gondoi is now known as zz_gondoi | 19:05 | |
| *** praneshp has joined #openstack-infra | 19:06 | |
| *** jistr has joined #openstack-infra | 19:06 | |
| *** marun has quit IRC | 19:07 | |
| *** e0ne has quit IRC | 19:09 | |
| *** che-arne has joined #openstack-infra | 19:10 | |
| *** Ryan_Lane has joined #openstack-infra | 19:10 | |
| *** Ryan_Lane has joined #openstack-infra | 19:11 | |
| *** ChuckC_ has joined #openstack-infra | 19:12 | |
| *** marun has joined #openstack-infra | 19:12 | |
| *** flaper87|afk is now known as flaper87 | 19:14 | |
| reed | Ryan_Lane, is there a staging server for the wiki? how do you test patches on that? | 19:16 |
| *** Clara_Zieme has joined #openstack-infra | 19:19 | |
| *** timo has joined #openstack-infra | 19:19 | |
| *** timo is now known as Guest97741 | 19:19 | |
| *** lttrl has joined #openstack-infra | 19:19 | |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Simport - Simple python import library (not entry-point based) https://review.openstack.org/121108 | 19:20 |
| *** laxity has quit IRC | 19:20 | |
| *** laxity has joined #openstack-infra | 19:21 | |
| *** Azher has quit IRC | 19:25 | |
| *** Clara_Zieme has quit IRC | 19:25 | |
| *** emagana has quit IRC | 19:26 | |
| *** HenryG is now known as HenryG_afk | 19:27 | |
| *** sabeen has joined #openstack-infra | 19:27 | |
| *** emagana has joined #openstack-infra | 19:27 | |
| *** mmaglana has joined #openstack-infra | 19:30 | |
| *** emagana has quit IRC | 19:32 | |
| *** e0ne has joined #openstack-infra | 19:32 | |
| *** mikeym has joined #openstack-infra | 19:35 | |
| openstackgerrit | A change was merged to openstack/requirements: Upgrade django_openstack_auth to version 1.1.7 for Horizon https://review.openstack.org/123101 | 19:36 |
| *** lcheng has joined #openstack-infra | 19:36 | |
| *** otherwiseguy has joined #openstack-infra | 19:38 | |
| *** nadya_ has quit IRC | 19:38 | |
| *** VijayTripathi1 has joined #openstack-infra | 19:39 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Line-by-line dib logs https://review.openstack.org/123528 | 19:39 |
| clarkb | fungi: ^ woot I think we can restart once that is installed | 19:40 |
| *** VijayTripathi has quit IRC | 19:40 | |
| *** marun has quit IRC | 19:40 | |
| fungi | w00tx0rz | 19:40 |
| fungi | and junk | 19:41 |
| *** VijayTripathi has joined #openstack-infra | 19:41 | |
| *** VijayTripathi1 has quit IRC | 19:43 | |
| *** zz_gondoi is now known as gondoi | 19:44 | |
| *** lcheng_ has joined #openstack-infra | 19:45 | |
| *** esker has quit IRC | 19:46 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add direct-release for training-guides and ha-guide https://review.openstack.org/123558 | 19:47 |
| *** lcheng has quit IRC | 19:47 | |
| *** esker has joined #openstack-infra | 19:48 | |
| *** juzuluag has quit IRC | 19:48 | |
| *** e0ne has quit IRC | 19:48 | |
| *** ZZelle_ has joined #openstack-infra | 19:49 | |
| *** mikeym has quit IRC | 19:51 | |
| *** mikeym has joined #openstack-infra | 19:51 | |
| *** mikeym has quit IRC | 19:51 | |
| *** mikeym has joined #openstack-infra | 19:51 | |
| *** mikeym has quit IRC | 19:51 | |
| *** mikeym has joined #openstack-infra | 19:52 | |
| *** mikeym has quit IRC | 19:52 | |
| *** mikeym has joined #openstack-infra | 19:52 | |
| *** mikeym has quit IRC | 19:52 | |
| Ryan_Lane | reed: no staging | 19:53 |
| *** mikeym has joined #openstack-infra | 19:53 | |
| *** mikeym has quit IRC | 19:53 | |
| Ryan_Lane | reed: http://jerodsanto.net/drop/test-in-production.png | 19:53 |
| *** MaxV has quit IRC | 19:53 | |
| clarkb | fyi git05 had OOMed again | 19:54 |
| clarkb | hrm actually was it the puppetmaster? | 19:54 |
| clarkb | ya its the puppet master | 19:54 |
| fungi | okay, that's at least twice. we should have it in cacti since the first occurrence so we can analyze the envelope | 19:55 |
| fungi | there were ruby processes using all the memories | 19:55 |
| clarkb | I restarted apache on puppetmaster | 19:55 |
| clarkb | since the ruby porcesses are apache things | 19:55 |
| fungi | huh. maybe we never approved the change to add it to cacti, or maybe that didn't take effect for some reason. i don't see it in the index | 19:56 |
| clarkb | probably just missed it. we should correct that | 19:56 |
| *** rmcall has quit IRC | 19:56 | |
| fungi | well, i wrote one and put it up for review after the initial oom | 19:57 |
| *** tonytan4ever has joined #openstack-infra | 19:57 | |
| fungi | i also rebooted the server the previous time, in case the oom killer also took out anything else of value in its frenzy | 19:57 |
| *** emagana has joined #openstack-infra | 19:57 | |
| *** emagana has quit IRC | 19:58 | |
| *** emagana has joined #openstack-infra | 19:59 | |
| *** dannywilson has quit IRC | 20:00 | |
| *** dane_leblanc has quit IRC | 20:00 | |
| *** esker has quit IRC | 20:01 | |
| *** kgiusti has quit IRC | 20:01 | |
| *** Sincler has joined #openstack-infra | 20:01 | |
| clarkb | jhesketh: I just approved your swift logs change | 20:01 |
| clarkb | jhesketh: should be able to start collecting some data on that real soon now | 20:02 |
| jhesketh | Oh Shiny :-) | 20:02 |
| jhesketh | Thanks | 20:02 |
| *** rlandy has quit IRC | 20:02 | |
| fungi | i'm going to self-approve stuff in my https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:puppet3,n,z stack since it already has at least 2x +2 votes | 20:02 |
| fungi | this is the outstanding puppet 3 related cleanup | 20:02 |
| clarkb | fungi: kk | 20:02 |
| clarkb | I am watching puppet run now to make sure it doesn't continue to die horribly | 20:02 |
| *** dane_leblanc has joined #openstack-infra | 20:03 | |
| *** juzuluag has joined #openstack-infra | 20:03 | |
| fungi | however it does seem the change to add puppetmaster.o.o to cacti merged weeks ago, so not sure why it didn't actually show up | 20:03 |
| clarkb | though I should make a sandwich | 20:03 |
| fungi | that's what sudo is for | 20:03 |
| *** bhuvan_ has joined #openstack-infra | 20:03 | |
| *** emagana has quit IRC | 20:04 | |
| clarkb | looks like it is running though | 20:05 |
| ianw | mordred / clarkb : didn't get to it in meeting, but i've started looking at centos7 builds https://review.openstack.org/#/c/123343/ (Initial centos7 support for build-image.sh) | 20:05 |
| *** VijayTripathi1 has joined #openstack-infra | 20:05 | |
| *** VijayTripathi has quit IRC | 20:05 | |
| *** rmcall has joined #openstack-infra | 20:05 | |
| *** baoli has joined #openstack-infra | 20:05 | |
| clarkb | ianw: why do we need to special case so heavily in build-image? | 20:06 |
| clarkb | ianw: don't the env vars cover the variance we need? | 20:06 |
| *** bhuvan has quit IRC | 20:07 | |
| *** VijayTripathi has joined #openstack-infra | 20:08 | |
| ianw | clarkb: i'm not sure every different platform agrees on what those DIB_* variables mean | 20:08 |
| ianw | clarkb: there's also some schism with DISTO and DISTRO_NAME that i'm looking at | 20:09 |
| *** derekh has quit IRC | 20:09 | |
| clarkb | ianw: I don't follow. DISTO=centos RELEASE=7 | 20:09 |
| *** VijayTripathi1 has quit IRC | 20:09 | |
| mordred | ianw: I agree with clark | 20:09 |
| mordred | dib already groks different things like this, which is what the DIB_ vars are for | 20:09 |
| mordred | we should not need any logic in build-image.sh - or else there is a bug somewhere else | 20:10 |
| *** dprince has quit IRC | 20:11 | |
| *** marcoemorais has quit IRC | 20:11 | |
| *** sabeen has quit IRC | 20:11 | |
| *** gondoi is now known as zz_gondoi | 20:11 | |
| *** hashar has quit IRC | 20:11 | |
| *** andreykurilin_ has quit IRC | 20:12 | |
| ianw | clarkb / mordred : ok, i think rhel and centos will need some reworking, i'll look into it | 20:12 |
| *** andreykurilin_ has joined #openstack-infra | 20:12 | |
| *** jistr has quit IRC | 20:13 | |
| *** sabeen has joined #openstack-infra | 20:13 | |
| *** esker has joined #openstack-infra | 20:14 | |
| *** maxbit_ has joined #openstack-infra | 20:14 | |
| *** wenlock has joined #openstack-infra | 20:14 | |
| *** AJaeger_ has quit IRC | 20:15 | |
| *** emagana has joined #openstack-infra | 20:16 | |
| *** james_li has joined #openstack-infra | 20:16 | |
| *** JayJ has quit IRC | 20:18 | |
| *** radez is now known as radez_g0n3 | 20:18 | |
| james_li | Hi can someone take a look at: https://review.openstack.org/#/c/122782/ ? | 20:18 |
| *** JayJ has joined #openstack-infra | 20:18 | |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/nodepool: Reset the client object before creating the image https://review.openstack.org/74943 | 20:19 |
| morganfainberg | how often does the translation proposal job run? | 20:19 |
| morganfainberg | i *think* keystone has an outstanding translation left before we can call it RC ready. | 20:20 |
| anteaya | AJaeger: ^ | 20:20 |
| anteaya | do you know? | 20:20 |
| morganfainberg | anteaya, AJaeger thanks! | 20:21 |
| morganfainberg | unfortunately we had 2 instances of *needed* new exceptions :( | 20:21 |
| morganfainberg | (mail to i18n list was sent) | 20:21 |
| * morganfainberg grumbles about icky release blocking bugs. | 20:22 | |
| *** esker has quit IRC | 20:26 | |
| *** gokrokve has quit IRC | 20:26 | |
| *** gokrokve has joined #openstack-infra | 20:26 | |
| *** arnaud has quit IRC | 20:27 | |
| clarkb | morganfainberg: we push up to transifex on every merge | 20:28 |
| clarkb | morganfainberg: we pull from transifex and propose to gerrit once a day | 20:28 |
| fungi | morganfainberg: they run daily, but are skipped if either 1) there's nothing new to propose, or 2) an existing change has been approved and is being gated | 20:28 |
| morganfainberg | clarkb, ok so i should see a propssal today. | 20:28 |
| clarkb | fungi: ok I have consumed sandwiches and potato salad | 20:28 |
| morganfainberg | clarkb, cool. | 20:28 |
| clarkb | morganfainberg: yup | 20:28 |
| clarkb | fungi: you want to do this nodepool restart? | 20:28 |
| morganfainberg | clarkb, i think that'll be the last thing we need :) | 20:29 |
| fungi | clarkb: sure, i'm on it | 20:29 |
| morganfainberg | clarkb, wooot | 20:29 |
| clarkb | fungi: I can do it too, just you had volunteered earlier | 20:29 |
| fungi | clarkb: i can restart it now if you're ready | 20:29 |
| clarkb | I am ready | 20:29 |
| fungi | it's restarted | 20:31 |
| clarkb | hrm OSError: [Errno 2] No such file or directory was thrown by dib | 20:31 |
| clarkb | or was it nodepool | 20:31 |
| fungi | raise child_exception | 20:31 |
| fungi | so dib? | 20:31 |
| clarkb | gah I bet it was the exec because the path that nodepool runs under doesn't include /usr/local/bin ? | 20:32 |
| *** amcrn has quit IRC | 20:32 | |
| *** VijayTripathi1 has joined #openstack-infra | 20:32 | |
| fungi | checking | 20:32 |
| *** gokrokve has quit IRC | 20:32 | |
| *** gokrokve has joined #openstack-infra | 20:33 | |
| fungi | yep | 20:33 |
| fungi | sudo cat /proc/21054/environ|strings|grep PATH | 20:33 |
| fungi | PATH=/sbin:/usr/sbin:/bin:/usr/bin | 20:33 |
| openstackgerrit | Ramy Asselin proposed a change to openstack-infra/devstack-gate: Add a pre-clean and post-clean hooks. https://review.openstack.org/122896 | 20:33 |
| openstackgerrit | Ramy Asselin proposed a change to openstack-infra/devstack-gate: Refactor the hook logic into a reusable function https://review.openstack.org/123578 | 20:33 |
| clarkb | /opt/nodepool_dib does exist and has the appropriate perms so I think is fine | 20:33 |
| clarkb | fungi: can we just cat into that file to fix this or does that really make /proc unhappy? | 20:33 |
| clarkb | fungi: as an alternative we could update the init script | 20:34 |
| *** VijayTripathi has quit IRC | 20:34 | |
| *** dangers is now known as dangers_away | 20:34 | |
| fungi | i would be uneasy about trying to munge the environment of a running process. children may need copies | 20:34 |
| *** todd_dsm has quit IRC | 20:34 | |
| clarkb | fungi: what do you think then? update the init script? | 20:37 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Initial centos7 support for build-image.sh https://review.openstack.org/123343 | 20:37 |
| fungi | i think we need to fix it in puppet | 20:37 |
| ianw | clarkb / mordred : ^ more what you expected? | 20:37 |
| clarkb | ya, I am updating the init script there now | 20:37 |
| fungi | but yes, init script | 20:37 |
| fungi | would be the place to append to the path | 20:38 |
| *** boden has quit IRC | 20:38 | |
| *** boden has joined #openstack-infra | 20:38 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Put /usr/local/bin in nodepool PATH for dib https://review.openstack.org/123580 | 20:39 |
| clarkb | fungi: jeblair mordred ^ | 20:39 |
| *** dannywilson has joined #openstack-infra | 20:39 | |
| clarkb | ianw: no we still have a case statement. | 20:40 |
| clarkb | ianw: that breaks precise and fedora too | 20:40 |
| clarkb | both of which did work at some point | 20:40 |
| *** baoli has quit IRC | 20:41 | |
| clarkb | ianw: I think it should be light weight enough to just work with arbitrary things if we feed it the right values | 20:41 |
| clarkb | and calling scope should worry about what those values are not the build image script | 20:41 |
| *** baoli has joined #openstack-infra | 20:42 | |
| *** dannywilson has quit IRC | 20:42 | |
| *** dannywilson has joined #openstack-infra | 20:42 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add channel set permission for IRC operators https://review.openstack.org/123532 | 20:43 |
| *** boden has quit IRC | 20:43 | |
| ianw | clarkb: my thought was to make it easier for someone trying to build an image to get it done, you'd have to grep around in d-i-b to figure out the correct DIB_RELEASE to set for centos7 because this script doesn't allow for using the default as-is | 20:44 |
| *** todd_dsm has joined #openstack-infra | 20:45 | |
| *** denis_makogon has quit IRC | 20:46 | |
| *** denis_makogon_ is now known as denis_makogon | 20:46 | |
| *** dmakogon_ has joined #openstack-infra | 20:46 | |
| *** marcoemorais has joined #openstack-infra | 20:46 | |
| clarkb | ianw: but it is removing support for other images | 20:46 |
| *** dannywilson has quit IRC | 20:46 | |
| *** zz_gondoi is now known as gondoi | 20:48 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add docs for the uncategorized page https://review.openstack.org/122293 | 20:50 |
| *** baoli has quit IRC | 20:51 | |
| *** baoli has joined #openstack-infra | 20:51 | |
| *** andreykurilin_ has quit IRC | 20:54 | |
| ianw | clarkb: ok; DIB_RELEASE is a bit funny because ubuntu is using it as the major-version (trusty, precise, etc) and centos7 has its major version in the name, and is using DIB_RELEASE as i guess the minor version | 20:56 |
| *** dannywilson has joined #openstack-infra | 20:56 | |
| openstackgerrit | David Lenwell proposed a change to openstack-infra/infra-manual: Add an introduction paragraph to the developer guide https://review.openstack.org/107024 | 20:57 |
| openstackgerrit | David Lenwell proposed a change to openstack-infra/infra-manual: Changed width of the image at the top of the page https://review.openstack.org/108867 | 20:57 |
| *** VijayTripathi has joined #openstack-infra | 21:00 | |
| *** zns has quit IRC | 21:00 | |
| *** gokrokve has quit IRC | 21:01 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 21:01 | |
| *** VijayTripathi1 has quit IRC | 21:02 | |
| *** dkliban is now known as dkliban_afk | 21:02 | |
| *** baoli has quit IRC | 21:03 | |
| *** baoli has joined #openstack-infra | 21:03 | |
| *** dannywilson has quit IRC | 21:03 | |
| *** dannywilson has joined #openstack-infra | 21:04 | |
| *** shashankhegde has quit IRC | 21:05 | |
| anteaya | number of nodes in delete state just increases substantially | 21:05 |
| anteaya | increased | 21:05 |
| *** marcoemorais has quit IRC | 21:06 | |
| *** marcoemorais1 has joined #openstack-infra | 21:06 | |
| *** bswartz has quit IRC | 21:06 | |
| fungi | anteaya: yeah, it happened when we restarted nodepool since it can't trust the state of formerly building nodes and so marks them all delete | 21:06 |
| anteaya | ah | 21:07 |
| *** shashankhegde has joined #openstack-infra | 21:07 | |
| clarkb | fungi: thank you for approving that change | 21:07 |
| *** wenlock has quit IRC | 21:07 | |
| fungi | i figure we can try with another restart shortly once nodepool has a chance to catch its breath | 21:08 |
| *** aysyanne has quit IRC | 21:08 | |
| *** dannywilson has quit IRC | 21:08 | |
| clarkb | fungi: with that change in or just applied by hand? | 21:08 |
| fungi | if it hasn't merged by then, applied by hand | 21:08 |
| clarkb | sounds good | 21:09 |
| clarkb | dstufft: can you weigh in on the most recent activity on https://bitbucket.org/pypa/bandersnatch/issue/47/simple-index-synced-for-package-after | 21:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Turn on swift log uploading for infra jobs https://review.openstack.org/122154 | 21:12 |
| *** dane_leblanc has quit IRC | 21:12 | |
| *** stevemar has quit IRC | 21:13 | |
| openstackgerrit | A change was merged to openstack-infra/config: Update default server value to new puppet master https://review.openstack.org/121638 | 21:14 |
| dstufft | clarkb: ok | 21:15 |
| clarkb | dstufft: just want to make sure that I am not misremembering how this stuff works | 21:16 |
| clarkb | also thank you | 21:16 |
| *** dustins has quit IRC | 21:17 | |
| *** wenlock has joined #openstack-infra | 21:17 | |
| krotscheck | What should I do about projects that are in the openstack-ci group, however also have their own launchpad project? | 21:18 |
| krotscheck | Example: Zuul | 21:18 |
| *** baoli has quit IRC | 21:18 | |
| *** baoli has joined #openstack-infra | 21:18 | |
| clarkb | krotscheck: wouldn't they get their own prjoect in storyboard with teh bugs there? | 21:19 |
| krotscheck | clarkb: Depends on whether any bugs against zuul have been filed against openstack-ci. | 21:20 |
| clarkb | krotscheck: its possible but they should probably be moved (either as part of the sync or we can move them by hand) | 21:20 |
| clarkb | er move them by hand in lp first | 21:21 |
| krotscheck | clarkb: Gotcha | 21:21 |
| fungi | clarkb: i went ahead and commented with a link to a similar issue i saw recently | 21:21 |
| clarkb | fungi: thanks | 21:22 |
| clarkb | krotscheck: don't want to overcomplicate the move though. so simple is better imo | 21:22 |
| *** vhoward has left #openstack-infra | 21:24 | |
| *** tonytan4ever has quit IRC | 21:24 | |
| openstackgerrit | Ramy Asselin proposed a change to openstack-infra/devstack-gate: Add a pre-clean and post-clean hooks. https://review.openstack.org/122896 | 21:24 |
| jeblair | freeze email sent | 21:24 |
| clarkb | fungi: we are 45 minutes in and 123580 hasn't run any jobs yet | 21:24 |
| dstufft | clarkb: fungi sadly part of this is just that the mirroring protocol isn't internally consistent | 21:25 |
| dstufft | it makes a bunch of HTTP requests and assumes that the world will be consistent during all of them | 21:25 |
| *** melwitt has joined #openstack-infra | 21:25 | |
| clarkb | dstufft: :/ | 21:25 |
| clarkb | jeblair: I see it | 21:25 |
| *** SumitNaiksatam has joined #openstack-infra | 21:26 | |
| clarkb | fungi: so not sure if we possibly want to move ahead with applying it by hand for now | 21:26 |
| dstufft | clarkb: it's basically wget --mirror with some extra smarts to optimize it | 21:26 |
| clarkb | dstufft: right it doesn't build indexes, it takes whatever the remote side has and uses it | 21:26 |
| jeblair | clarkb, fungi: yeah, let's just do that by hand on the server and puppet will catch up later | 21:26 |
| fungi | clarkb: yep, i'll disable puppet on nodepool temporarily while we wait for that change to worm its way through the gate, hand-apply it on the server and give it a restart | 21:26 |
| clarkb | its a similar problem to the rax apt mirrors | 21:26 |
| clarkb | fungi: ++ ty | 21:27 |
| dstufft | I want to make it a journaled thing, where instead of just taking wahtever, it applies transactions to itself and each trasnaction has to apply successfully or be rolled back | 21:27 |
| dstufft | (this sounds more complicated than it is) | 21:27 |
| *** koolhead17 has quit IRC | 21:27 | |
| dstufft | brb gotta vpn and will get knocked off irc | 21:27 |
| *** koolhead17 has joined #openstack-infra | 21:28 | |
| openstackgerrit | A change was merged to openstack-infra/config: Cease using ci-puppetmaster.openstack.org https://review.openstack.org/121642 | 21:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Update config documentation to refer to new master https://review.openstack.org/121644 | 21:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Use the current Puppet master when launching https://review.openstack.org/121654 | 21:28 |
| fungi | jeblair: clarkb: restarted now | 21:29 |
| fungi | jeblair: clarkb: sudo strings /proc/31047/environ | grep PATH | 21:29 |
| fungi | jeblair: clarkb: PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin | 21:29 |
| fungi | not seeing the oserror exceptions thrown any longer | 21:30 |
| clarkb | fungi: cool I am currently grepping for where it started | 21:30 |
| *** hemna has joined #openstack-infra | 21:30 | |
| *** changbl_ has quit IRC | 21:31 | |
| clarkb | fungi: and ya I see no error | 21:31 |
| clarkb | and there is stuff in the image.log \o/ | 21:32 |
| clarkb | now we wait patiently | 21:32 |
| *** koolhead17 has quit IRC | 21:32 | |
| krotscheck | So, I think I’ve found 13 infra projects that need to be migrated. | 21:33 |
| fungi | 2014-09-23 21:32:43,507 WARNING nodepool.NodePool: Missing image devstack-trusty-dib on hpcloud-b1 | 21:33 |
| fungi | et cetera | 21:33 |
| krotscheck | I’ve got an etherpad with the process - https://etherpad.openstack.org/p/infra-storyboard-migration | 21:33 |
| fungi | good sign | 21:33 |
| *** andreykurilin_ has joined #openstack-infra | 21:33 | |
| clarkb | fungi: ya and if tail -f /var/log/nodepool/image/image.log jeblair's logging change is working well | 21:33 |
| *** mmedvede has quit IRC | 21:33 | |
| clarkb | it is currently building its cache | 21:33 |
| krotscheck | I’m a little leery about moving the openstack-org group. Is openstackid a thing that we want to migrate? | 21:33 |
| fungi | i see that | 21:34 |
| fungi | krotscheck: openstack-org in lp is mainly covering the www site which is outside infra's sphere of control | 21:34 |
| jeblair | openstackid should be moved | 21:34 |
| jeblair | krotscheck: the list in step3 doesn't look quite right | 21:35 |
| * jeblair edits | 21:35 | |
| *** juzuluag has quit IRC | 21:35 | |
| clarkb | this smart caching stuff should be a package in that new bash packager thing | 21:36 |
| krotscheck | Ok, so we have both openstack-org AND openstackid as a project on launchpad. I’m going to assume that the latter should be moved, but the former shouldn't. | 21:36 |
| fungi | bpkg abscond dib-cacher | 21:36 |
| jeblair | krotscheck: i believe that's correct | 21:36 |
| *** mfer has quit IRC | 21:37 | |
| jeblair | okay i think the list in step3 is correct now | 21:37 |
| *** james_li has quit IRC | 21:37 | |
| krotscheck | Cool | 21:37 |
| krotscheck | Let’s see what happens when I try to run the migration :D | 21:38 |
| jeblair | krotscheck: and made one change to step4 (openstack-ci -> config) | 21:38 |
| krotscheck | Got it | 21:38 |
| dstufft | clarkb: fungi ok back, but yea, at the end of the day bandersnatch needs to decide what it's going to be consistent with primarily and operate on that I think | 21:39 |
| jeblair | clarkb: where are the dib logs going? | 21:39 |
| clarkb | jeblair: the image log | 21:39 |
| clarkb | jeblair: /var/log/nodepool/image/image.log | 21:39 |
| jeblair | excellent | 21:39 |
| *** baoli has quit IRC | 21:39 | |
| fungi | that's where i would expect to find them, so working well | 21:40 |
| *** baoli has joined #openstack-infra | 21:40 | |
| *** andreaf has quit IRC | 21:41 | |
| *** mriedem has quit IRC | 21:41 | |
| *** andreaf has joined #openstack-infra | 21:42 | |
| *** sarob_away is now known as sarob | 21:43 | |
| *** amcrn has joined #openstack-infra | 21:44 | |
| fungi | looks like it's puppeting now | 21:44 |
| *** dannywilson has joined #openstack-infra | 21:44 | |
| krotscheck | So, launchpadlib’s got an unverified/unsecure dependency. Is it appropriate for me to not include that in the requirements.txt? | 21:45 |
| clarkb | krotscheck: that won't help fix the problem. you have to tell pip to really download that dep | 21:45 |
| jeblair | i think we should avoid putting launchpadlib in requirements.txt | 21:45 |
| jeblair | we should probably just document it as an optional dependency, and note that you'll need it if you want to run the migration script | 21:46 |
| clarkb | and agreed for whta is a couple time thing we probably don't need to hard require on it | 21:46 |
| jeblair | (because when you step back, storyboard depending on launchpadlib is not what we want to be doing :) | 21:46 |
| clarkb | jeblair: ++ | 21:46 |
| krotscheck | So it’s ok that it’s not there, we just need to make a note that it needs to be installed before running the script. | 21:47 |
| krotscheck | Chekc | 21:47 |
| fungi | i think that's a totally reasonable expectation | 21:47 |
| jeblair | yep. we may want to encode this in puppet-storyboard though | 21:47 |
| jeblair | i think that would be nice, but i could be talked out of that :) | 21:47 |
| clarkb | fyi hpcloud east is having some trouble with nova api | 21:48 |
| clarkb | its returning 500s | 21:48 |
| jeblair | (since this is a one-time (well, two-time) thing) | 21:48 |
| anteaya | can someone re-enable 9846 ibmsdnve IBM SDN-VE CI ibmsdnve@us.ibm.com please? | 21:48 |
| *** bhuvan_ has quit IRC | 21:48 | |
| krotscheck | puppet-storyboard doesn’t have a migration target. | 21:48 |
| clarkb | anteaya: ya | 21:48 |
| *** dannywilson has quit IRC | 21:48 | |
| anteaya | thanks | 21:48 |
| *** pcm_ has quit IRC | 21:48 | |
| jeblair | tteggel: ^ clarkb's message about hpcloud east nova api 500s | 21:48 |
| clarkb | jeblair: it looks like it is a known thing according the the status page | 21:49 |
| clarkb | anteaya: done | 21:49 |
| anteaya | clarkb: thank you | 21:49 |
| clarkb | no problem | 21:49 |
| *** todd_dsm has quit IRC | 21:50 | |
| *** mbacchi has quit IRC | 21:51 | |
| *** bhuvan has joined #openstack-infra | 21:52 | |
| *** todd_dsm has joined #openstack-infra | 21:52 | |
| *** maxbit has joined #openstack-infra | 21:53 | |
| *** maxbit_ has quit IRC | 21:54 | |
| *** baoli has quit IRC | 21:54 | |
| *** baoli has joined #openstack-infra | 21:55 | |
| *** todd_dsm has quit IRC | 21:56 | |
| clarkb | looks like we are converting to qcow2 now :) | 21:57 |
| *** james_li has joined #openstack-infra | 21:57 | |
| lifeless | man | 21:58 |
| lifeless | we just can't have nice things | 21:58 |
| *** dannywilson has joined #openstack-infra | 21:58 | |
| lifeless | http://paste.ubuntu.com/8413835/ | 21:58 |
| *** ayoung has joined #openstack-infra | 21:58 | |
| lifeless | mordred: ^ | 21:58 |
| jeblair | mordred, fungi: https://anonscm.debian.org/cgit/tasksel/tasksel.git/commit/?id=dce99f5f8d84e4c885e6beb4cc1bb5bb1d9ee6d7 | 21:59 |
| ayoung | jeblair, lifeless mordred can one of you guys pull the trigger on https://review.openstack.org/#/c/120261/ | 21:59 |
| lifeless | dhellmann: mordred: do we have consensus on the semver ting? | 21:59 |
| lifeless | https://bugs.launchpad.net/pbr/+bug/1373148 | 22:00 |
| uvirtbot | Launchpad bug 1373148 in pbr "setuptools import loop" [Undecided,New] | 22:00 |
| lifeless | ayoung: lookin | 22:00 |
| lifeless | ayoung: I'm not core here, sorry | 22:00 |
| clarkb | I should put "I will not answer media inquirues about this commit" in all of my commits | 22:00 |
| *** wenlock has quit IRC | 22:00 | |
| ayoung | lifeless, thanks anyway | 22:00 |
| jeblair | ayoung, morganfainberg: do you totally pinky swear that dolphm will +1 120310 in the governance repo? | 22:00 |
| jeblair | i ask because we have gotten bit by that one before :) | 22:01 |
| ayoung | jeblair, this was done at his direction | 22:01 |
| ayoung | and ttx just said it was ok in #openstack-dev | 22:01 |
| jeblair | okay, wfm. no takebacks. | 22:01 |
| ayoung | jeblair, thanks | 22:01 |
| morganfainberg | jeblair, shouldn't be an issue | 22:01 |
| *** harlowja is now known as harlowja_away | 22:01 | |
| ttx | jeblair: arguably dolph delegated PTL to morgan | 22:02 |
| ttx | while he is away | 22:02 |
| ttx | so we can run with that | 22:02 |
| morganfainberg | jeblair, ttx, and i'm going to say "it's fine" with that authority, :) | 22:02 |
| ttx | yeahn governance deadlock solved | 22:02 |
| jeblair | awesome, thanks, Acting PTL morganfainberg! | 22:02 |
| morganfainberg | jeblair, anytime! | 22:02 |
| * ttx can go to bed peacefully | 22:02 | |
| morganfainberg | ttx sleep well | 22:02 |
| jeblair | +3 | 22:03 |
| ttx | morganfainberg: thx to you I will! | 22:03 |
| *** gondoi is now known as zz_gondoi | 22:03 | |
| ayoung | jeblair, thanks. | 22:03 |
| *** harlowja_away is now known as harlowja | 22:03 | |
| *** wenlock has joined #openstack-infra | 22:03 | |
| *** miqui has quit IRC | 22:04 | |
| clarkb | jeblair: fungi: image built appears to be uloading now | 22:04 |
| clarkb | 5 x 3GB = #WINNING | 22:04 |
| clarkb | not sure how long that will take | 22:04 |
| mtreinish | clarkb: not 15GB? | 22:04 |
| fungi | jeblair: actually the default desktop choice in debian doesn't worry me, other than possibly as a sign of things to come (though worth noting that it used to be gnome before it was "briefly" switched, arguably by fiat, to xfce) | 22:05 |
| clarkb | mtreinish: no 15GB autoagmically becomes #winning when copying data from one cloud provider to another | 22:05 |
| mtreinish | clarkb: heh, makes sense to me. | 22:05 |
| clarkb | fungi: gnome3 ? | 22:05 |
| fungi | gnome-whatever-i-don't-use-it-anyway | 22:05 |
| jeblair | fungi: yeah, i find the fact that systemd is partially responsible for that very interesting | 22:06 |
| clarkb | oh that reminds me | 22:06 |
| fungi | agreed | 22:06 |
| mtreinish | fungi: switch to arch, then you don't have to worry about fighting against a default desktop env... | 22:06 |
| jeblair | if anyone wondered what the actual _mechanics_ of systemd taking everything over, this is what it will look like | 22:06 |
| *** pblaho has joined #openstack-infra | 22:06 | |
| *** gokrokve has joined #openstack-infra | 22:06 | |
| ayoung | "Whenever people are carrying pitchforks these days it seems to be due to Lennart" one of my favorite quotes | 22:06 |
| fungi | mtreinish: i install essential packages only and then just pick the bits i need to build a desktop environment anyway | 22:06 |
| *** pblaho has quit IRC | 22:06 | |
| *** flaper87 is now known as flaper87|afk | 22:07 | |
| jeblair | clarkb: i don't think we have any interesting log messages about uploads, or am i missing them? | 22:07 |
| clarkb | fungi: mordred jeblair http://aceattorney.sparklin.org/jeu.php?id_proces=57684 | 22:07 |
| *** bknudson has quit IRC | 22:07 | |
| fungi | mtreinish: it's not like debian forces a desktop on anybody | 22:07 |
| clarkb | jeblair: I don't think so. glaneclient is pretty quiet | 22:07 |
| jeblair | clarkb: do we even log start/stop for that? | 22:07 |
| clarkb | jeblair: hrm we may not /me looks | 22:07 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Initial centos7 support for build-image.sh https://review.openstack.org/123343 | 22:08 |
| *** JayJ has quit IRC | 22:08 | |
| ianw | clarkb: yay, that builds pretty quick | 22:08 |
| *** armax has quit IRC | 22:08 | |
| *** zehicle_at_dell has quit IRC | 22:09 | |
| clarkb | jeblair: I am not seeing any logs in the code that would be super useful | 22:09 |
| *** otherwiseguy has quit IRC | 22:09 | |
| fungi | i'm gonna go grab some dinner, then come back to find out if we got thoroughly dibbed | 22:10 |
| clarkb | jeblair: we haev a "building image" and "Image.*is ready" pair of messages | 22:10 |
| *** julim has quit IRC | 22:11 | |
| jeblair | clarkb: right, building image is confusing, actually. | 22:11 |
| clarkb | jeblair: yeah, it should be uploading image or similar | 22:11 |
| jeblair | i uploads it, gets an id, logs that it is building, then waits for it to exist (?), then logs that it's ready | 22:11 |
| clarkb | fungi: you should play some debian ace attorney over dinner | 22:12 |
| jeblair | clarkb: i just noticed this: | 22:12 |
| jeblair | 2014-09-23 21:28:43,290 DEBUG nodepool.NodePool: Queued image building task for devstack-trusty-dib | 22:12 |
| jeblair | clarkb: that's logged every few tens of seconds | 22:12 |
| jeblair | clarkb: have we started a second build yet? i'm worried that nodepool now has a several-thousands deep queue | 22:13 |
| clarkb | jeblair: 2014-09-23 22:02:06,566 INFO nodepool.image.build.devstack-trusty-dib: Image file /opt/nodepool_dib/devstack-trusty-dib-1411507723.qcow2 created... is most recent thing in the image log | 22:13 |
| clarkb | I see no disk processes running | 22:14 |
| clarkb | jeblair: so that is probably noise? should be cleaned up though | 22:14 |
| mtreinish | clarkb: that ace attorney link is awesome | 22:14 |
| *** andreykurilin_ has quit IRC | 22:14 | |
| clarkb | mtreinish: I used it to tl;dr the situation back when it was going on | 22:15 |
| clarkb | had a good time | 22:15 |
| jeblair | 2014-09-23 21:28:27,426 INFO nodepool.DiskImageBuilderThread: Creating image: devstack-trusty-dib with filename /opt/nodepool_dib/devstack-trusty-dib-1411507707.qcow2 | 22:16 |
| jeblair | 2014-09-23 21:28:43,299 INFO nodepool.DiskImageBuilderThread: Creating image: devstack-trusty-dib with filename /opt/nodepool_dib/devstack-trusty-dib-1411507723.qcow2 | 22:16 |
| jeblair | it keeps logging that too | 22:16 |
| mattoliverau | Morning | 22:16 |
| anteaya | morning mattoliverau | 22:16 |
| jeblair | clarkb: oh wait, that was all before the most recent build wasn't it? | 22:16 |
| jeblair | that must have been when we were looping and not functioning | 22:17 |
| openstackgerrit | A change was merged to openstack-infra/config: Add keystoneclient-kerberos repository https://review.openstack.org/120261 | 22:17 |
| jeblair | ayoung: ^ | 22:17 |
| lifeless | dhellmann: around ? | 22:17 |
| clarkb | jeblair: ya those timestamps are old | 22:17 |
| jeblair | 2014-09-23 22:17:04,082 INFO nodepool.DiskImageUpdater: Image devstack-trusty-dib-1411509742.template.openstack.org in hpcloud-b1 is ready | 22:17 |
| clarkb | jeblair: I think that is correct | 22:17 |
| jeblair | that one is new | 22:17 |
| clarkb | nodepool image-list corroborates | 22:17 |
| clarkb | b4 is ready too | 22:18 |
| *** baoli has quit IRC | 22:18 | |
| clarkb | now we should try booting at least one node because that is the min ready | 22:18 |
| *** baoli has joined #openstack-infra | 22:18 | |
| clarkb | Deficit: devstack-trusty-dib: 0 (start: 0 min: 0 ready: 0 capacity: 0) hrm maybe not until after we are not contrained for nodes? | 22:19 |
| clarkb | or maybe I missed the bit where it started and allocated 1 | 22:19 |
| clarkb | nope no dib images slaves building | 22:19 |
| clarkb | <AllocationGrant of 1 of devstack-trusty-dib from hpcloud-b4> there we go | 22:20 |
| clarkb | this is so exciting :P | 22:20 |
| clarkb | hopefully cloud init doesn't barf | 22:20 |
| *** marcusvrn has quit IRC | 22:21 | |
| ayoung | jeblair, so is there just a delay until I see it in this list http://git.openstack.org/cgit/openstack/ | 22:23 |
| *** cnesa15 has joined #openstack-infra | 22:23 | |
| clarkb | ayoung: yes, you need to wait for puppet to run which happens every 15 minutes | 22:23 |
| clarkb | ayoung: so about 2240UTC you should see it | 22:23 |
| ayoung | good...I'll check when I get home then. THanks all | 22:23 |
| *** otherwiseguy has joined #openstack-infra | 22:25 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Add centos7 image to nodepool d-i-b config https://review.openstack.org/123594 | 22:26 |
| *** denis_makogon has quit IRC | 22:26 | |
| *** zns has joined #openstack-infra | 22:28 | |
| clarkb | ok the slave is up and ACTIVE but doesn't have a floating ip yet | 22:28 |
| *** atiwari has quit IRC | 22:28 | |
| clarkb | nova show doesn't show anything amiss other than the floating ip thing | 22:28 |
| *** ayoung has quit IRC | 22:28 | |
| *** datsun180b has quit IRC | 22:29 | |
| arosen | Sorry I had asked this before but got stuck. Anyone know the configuration that's needed to be done to logs.openstack.org so that it lets browsers view the .gz files without having to download them? | 22:30 |
| *** openstackgerrit has quit IRC | 22:31 | |
| ianw | arosen: if it's a .txt.gz it should be displayed ... is that what you mean? | 22:31 |
| *** openstackgerrit has joined #openstack-infra | 22:32 | |
| ianw | arosen: i.e. the solution is probably to rename the file to .txt.gz in the devstack-gate cleanup scripts; that is what is mostly done in there ... which file? | 22:32 |
| clarkb | arosen: I believe its the encoding at https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/logs.vhost.erb#n24 | 22:32 |
| arosen | clarkb: thanks | 22:33 |
| clarkb | though it may be something in the wsgi app instead | 22:33 |
| anteaya | happy equinox | 22:34 |
| clarkb | we are not getting a floating ip on that node | 22:35 |
| jeblair | clarkb: it still shows as building in nodepool | 22:37 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Honour MANIFEST.in excludes https://review.openstack.org/123597 | 22:37 |
| clarkb | jeblair: ya | 22:37 |
| lifeless | dhellmann: dims: mordred: ^ :) | 22:37 |
| *** doug-fish has left #openstack-infra | 22:38 | |
| lifeless | bah, now ^ :) | 22:38 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Honour MANIFEST.in excludes https://review.openstack.org/123597 | 22:38 |
| *** bhuvan has quit IRC | 22:38 | |
| clarkb | thats a cool horizon bug. it is displaying a ton of negative items | 22:38 |
| clarkb | eventually it says displaying 0 though | 22:38 |
| *** ZZelle_ has quit IRC | 22:40 | |
| clarkb | also they are reverse sorted by uptime | 22:40 |
| clarkb | not sure if I prefer new nodes or old nodes at the head of the list | 22:40 |
| *** reed has quit IRC | 22:41 | |
| *** james_li has quit IRC | 22:42 | |
| *** mjturek has quit IRC | 22:42 | |
| *** bhuvan has joined #openstack-infra | 22:42 | |
| *** jgrimm is now known as zz_jgrimm | 22:45 | |
| clarkb | jeblair: 15.126.199.172 | 22:45 |
| clarkb | jeblair: that doesn't seem to have reflected in nodepool yet though | 22:45 |
| jeblair | it just errored | 22:46 |
| *** dmsimard is now known as dmsimard_away | 22:46 | |
| jeblair | grabbing the error now | 22:46 |
| clarkb | jeblair: ok | 22:46 |
| jeblair | IOError: [Errno 2] No such file or directory: '/var/lib/jenkins/.ssh/id_rsa' | 22:47 |
| jeblair | i'm guessing a nodepool config error | 22:47 |
| clarkb | oh crud did I set that up wrong? | 22:47 |
| jeblair | username: jenkins | 22:48 |
| jeblair | private-key: /home/nodepool/.ssh/id_rsa | 22:48 |
| jeblair | clarkb: i'm guessing you need to add that to the devstack-trusty-dib image stanzas | 22:48 |
| jeblair | though really | 22:48 |
| *** weshay has quit IRC | 22:48 | |
| clarkb | jeblair: oh ok | 22:48 |
| clarkb | so it must be defaulting to the jenkins path | 22:49 |
| jeblair | logically, that should be part of the diskimage definition, but i do not think that it is currently | 22:49 |
| jeblair | clarkb: correct | 22:49 |
| jeblair | that's the default value | 22:49 |
| clarkb | jeblair: did you want to make that change locally in nodepool.yaml on nodepool.o.o while puppet is disabled? I will push a change up to do it asap | 22:49 |
| jeblair | clarkb: will do | 22:49 |
| jeblair | done | 22:50 |
| *** dkranz has quit IRC | 22:51 | |
| jeblair | clarkb: 2368912 is building now; i don't know if the config change will affect it, or the next one. | 22:51 |
| jeblair | so if it fails, we should give it another chance | 22:51 |
| *** baoli has quit IRC | 22:51 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Set the nodepool user and key on dib images https://review.openstack.org/123603 | 22:52 |
| clarkb | jeblair: rgr and ^ | 22:52 |
| jeblair | clarkb: that's what i did. aprvd. | 22:53 |
| *** todd_dsm has joined #openstack-infra | 22:53 | |
| *** dims has quit IRC | 22:53 | |
| *** dims has joined #openstack-infra | 22:54 | |
| clarkb | jeblair: from the short tiem Iwas on the first node built it looked pretty sane | 22:55 |
| clarkb | but the node has been killed so have to wait for the next one to look again | 22:55 |
| *** bdpayne has quit IRC | 22:56 | |
| *** lcheng_ has quit IRC | 22:56 | |
| *** dims has quit IRC | 22:59 | |
| clarkb | jeblair: note the provider manager was restarted so a third node build has started | 23:00 |
| *** patrickeast has quit IRC | 23:00 | |
| *** patrickeast has joined #openstack-infra | 23:00 | |
| jeblair | cool, so 2369032 should be using the new config | 23:00 |
| *** wenlock has quit IRC | 23:01 | |
| clarkb | yup | 23:01 |
| clarkb | jeblair: 15.126.211.108 | 23:02 |
| *** sarob is now known as sarob_away | 23:02 | |
| *** armax has joined #openstack-infra | 23:03 | |
| clarkb | the git cache is there like we expect | 23:03 |
| clarkb | and the VM images and pip install thing are in /home/jenkins/cache | 23:04 |
| clarkb | jeblair: 2014-09-23 23:04:04,766 INFO nodepool.NodeLauncher: /etc/nodepool does not exist on node id: 2369032 | 23:05 |
| clarkb | jeblair: is the image build process expected to create that dir? | 23:05 |
| *** Guest97741 has quit IRC | 23:05 | |
| lifeless | ok, I'm declaring victory over namespaces | 23:06 |
| lifeless | NEXT | 23:06 |
| clarkb | I can confirm the /etc/nodepool dir does not exist. I will whip up a change to fix that | 23:06 |
| clarkb | we are sooooo close | 23:06 |
| *** zns has quit IRC | 23:06 | |
| *** dansmith has quit IRC | 23:06 | |
| *** emagana has quit IRC | 23:07 | |
| clarkb | hrm it isn't clea rto me who is supposed to create /etc/nodepook | 23:07 |
| *** marcoemorais1 has quit IRC | 23:07 | |
| *** sdake has quit IRC | 23:07 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack-infra/devstack-gate: Enable DIB service for Ironic https://review.openstack.org/122539 | 23:07 |
| *** marcoemorais has joined #openstack-infra | 23:08 | |
| *** emagana has joined #openstack-infra | 23:08 | |
| clarkb | oh wait stdout: bash: line 0: cd: /opt/nodepool-scripts: No such file or directory is the issue | 23:08 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added catch for OpenID Failures in Launchpad Import https://review.openstack.org/123605 | 23:08 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Sanitize title during launchpad import. https://review.openstack.org/123606 | 23:09 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Expanded length of storytag name. https://review.openstack.org/123607 | 23:09 |
| *** boden has joined #openstack-infra | 23:09 | |
| krotscheck | ^ With these three patches, I was able to import all of the infra projects listed here: https://etherpad.openstack.org/p/infra-storyboard-migration | 23:10 |
| *** dansmith has joined #openstack-infra | 23:10 | |
| clarkb | lifeless: if extra-data.d is mispelled as exta-data.d I assume that those run aprts won't run? | 23:10 |
| clarkb | if so that would explain why we didn't get these scrpts | 23:10 |
| * krotscheck still needs to fix indexing. | 23:10 | |
| fungi | so are we all dibbed up now? | 23:10 |
| krotscheck | Also, the launchpad api is not fast. | 23:10 |
| clarkb | fungi: almost. node boots are failing in the post boot nodepool check stage | 23:11 |
| clarkb | fungi: I think I know why though | 23:11 |
| lifeless | clarkb: indeed, we don't do closest-match checks :) | 23:11 |
| lifeless | clarkb: a lint to find unknown .d dirs would help you | 23:11 |
| clarkb | lifeless: I figured :) | 23:12 |
| clarkb | I am running the dib build locally to make sure there isn't a bug in that script that breaks the build | 23:12 |
| clarkb | should know in about 15 minutes | 23:12 |
| *** salv-orlando has quit IRC | 23:12 | |
| *** beekneemech has joined #openstack-infra | 23:13 | |
| *** bnemec has quit IRC | 23:13 | |
| *** andreaf has quit IRC | 23:13 | |
| *** andreaf has joined #openstack-infra | 23:14 | |
| fungi | yep, just finished catching up on scrollback excitement | 23:14 |
| *** andreaf has quit IRC | 23:15 | |
| *** boden has quit IRC | 23:15 | |
| *** bswartz has joined #openstack-infra | 23:15 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Rename nodepool exta-data.d to extra-data.d https://review.openstack.org/123609 | 23:17 |
| clarkb | don't approve that just yet. I want my local sanity check to finish first | 23:17 |
| clarkb | jeblair is the lack of an /etc/nodepool a problem? or does nodepool create that? or does one of the erady scripts do it? | 23:18 |
| clarkb | my grepping in the config repo wasn't super helpful in figureing that out | 23:19 |
| *** gokrokve_ has joined #openstack-infra | 23:19 | |
| krotscheck | mordred: So, with a bit of testing I discovered that we can get around the autoincrement thing by first doing an insert with an autoindex, and then doing an update to change the id. Will that be sufficient? | 23:19 |
| krotscheck | I’m not certain it is :/ | 23:19 |
| lifeless | krotscheck: autoincrement thing? | 23:19 |
| *** dims has joined #openstack-infra | 23:19 | |
| lifeless | oh look a dims :) | 23:20 |
| krotscheck | lifeless: Migrating launchpad to storyboard while preserving Launchpad ID’s on stories. | 23:20 |
| *** Sincler has quit IRC | 23:20 | |
| *** dannywilson has quit IRC | 23:20 | |
| *** sdake has joined #openstack-infra | 23:20 | |
| dims | hi lifeless looking at https://review.openstack.org/#/c/123597 | 23:20 |
| jeblair | clarkb: i believe the image build scripts are supposed to create it | 23:20 |
| lifeless | dims: \o/ | 23:20 |
| *** david-lyle has quit IRC | 23:20 | |
| *** dannywilson has joined #openstack-infra | 23:20 | |
| *** gokrokve has quit IRC | 23:21 | |
| *** dhellmann has quit IRC | 23:21 | |
| jeblair | clarkb: so i'm guessing we're missing an element to create that | 23:21 |
| *** salv-orlando has joined #openstack-infra | 23:21 | |
| jeblair | (this is why actual testing is important) | 23:21 |
| *** wenlock has joined #openstack-infra | 23:21 | |
| *** dhellmann has joined #openstack-infra | 23:21 | |
| clarkb | jeblair: ok I will write that change too | 23:21 |
| *** david-lyle has joined #openstack-infra | 23:21 | |
| *** dannywilson has quit IRC | 23:21 | |
| *** dannywilson has joined #openstack-infra | 23:22 | |
| jeblair | i think we need to get a lot better about people actually testing changes before they push them up. | 23:22 |
| *** otherwiseguy has quit IRC | 23:23 | |
| jeblair | because nothing we're running into needed to be found in production | 23:23 |
| lifeless | jeblair: could we have tests for this in CI ? | 23:23 |
| jeblair | lifeless: conceivably. they would take hours and hours to run. | 23:24 |
| jeblair | lifeless: and kind of depend on this system being in place to even think about doing them | 23:24 |
| *** otherwiseguy has joined #openstack-infra | 23:24 | |
| clarkb | jeblair: nodepool does host.ssh("make config dir", "sudo mkdir /etc/nodepool") | 23:25 |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:25 | |
| jeblair | clarkb: ah right. so before the scripts | 23:26 |
| ianw | jeblair: i've done building images and booting them, but running a full nodepool environment to test more is a really big commitment | 23:26 |
| *** david-lyle has quit IRC | 23:26 | |
| jeblair | clarkb: it can't do that with dib, so it'll need to be in elements | 23:26 |
| clarkb | jeblair: oh wait it does that in the image build | 23:26 |
| clarkb | jeblair: yup | 23:26 |
| clarkb | jeblair: I thought it did it for every node. /me adds to elements | 23:26 |
| jeblair | ianw: running nodepool is not a big commitment | 23:26 |
| *** signed8bit_ZZZzz has quit IRC | 23:26 | |
| jeblair | ianw: you need a database and a cloud provider | 23:27 |
| jeblair | ianw: it doesn't depend on anything else running | 23:27 |
| *** QuinnyPig has quit IRC | 23:27 | |
| fungi | in fact, i believe pabelanger runs his with a near-zero-footprint configuration | 23:27 |
| fungi | basically build images but set min-ready=0 and then only boot nodes when there are jobs to run | 23:28 |
| jeblair | fungi: you don't need to run jobs to test nodepool | 23:28 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Run the precise nonha job on tripleo projects https://review.openstack.org/122099 | 23:28 |
| jeblair | most of this is just testing that things build | 23:28 |
| fungi | jeblair: indeed, but just pointing out that you can run nodepool and tell it to build no nodes at all | 23:28 |
| jeblair | anyway, it would be great if people would test things locally. we have a lot of tooling and instruction to help out with that. | 23:29 |
| *** jpich has joined #openstack-infra | 23:30 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add a nodepool dib script to create /etc/nodepool https://review.openstack.org/123611 | 23:30 |
| *** dannywilson has quit IRC | 23:30 | |
| krotscheck | Can someone peek at the storyboard apache logs to see whether our new rabbitmq publisher is as robust as we want it to be? | 23:30 |
| clarkb | jeblair: I am testing 123611 and the extra-data.d rename now | 23:30 |
| clarkb | (though not with full nodepool, as I don't have that handy. going to check the resulting image) | 23:30 |
| *** dannywilson has joined #openstack-infra | 23:30 | |
| *** mmedvede has joined #openstack-infra | 23:31 | |
| clarkb | except I can't hit my test node anymore... | 23:31 |
| *** ayoung has joined #openstack-infra | 23:31 | |
| jeblair | i've run out of time today, and to be honest, considering the rather aggresive schedule we've chosen for the project-config rename, i'm probably out of time to work on dib at all until i get back from vacation | 23:32 |
| *** QuinnyPig has joined #openstack-infra | 23:33 | |
| clarkb | I think this is basically there... | 23:34 |
| jeblair | i thought that last night :( | 23:34 |
| clarkb | once the ready script can run and we have /etc/nodepool we should get ready nodes | 23:34 |
| clarkb | any changes at that point will hopefully only be related to image differences affecting jobs | 23:34 |
| *** Annetta_Nitzsche has joined #openstack-infra | 23:37 | |
| arosen | clarkb: I got the right vhost apache settings worrking now :) any idea bout getting the dates to be link able in the log files? | 23:39 |
| clarkb | arosen linkable dates requires the osloganalyze wsgi app thing | 23:41 |
| *** gargola has quit IRC | 23:41 | |
| clarkb | it rewrites the logs to html making them linkable | 23:41 |
| arosen | clarkb: ah okay cool | 23:42 |
| *** hdd has quit IRC | 23:43 | |
| *** dannywilson has quit IRC | 23:43 | |
| *** Annetta_Nitzsche has quit IRC | 23:44 | |
| openstackgerrit | A change was merged to openstack-infra/config: Put /usr/local/bin in nodepool PATH for dib https://review.openstack.org/123580 | 23:44 |
| *** Mmike has quit IRC | 23:48 | |
| *** ChuckC_ has quit IRC | 23:49 | |
| *** Margaretta_Gorcz has joined #openstack-infra | 23:49 | |
| *** ChuckC_ has joined #openstack-infra | 23:49 | |
| *** Mmike has joined #openstack-infra | 23:49 | |
| *** wenlock has quit IRC | 23:50 | |
| *** ryanpetrello has quit IRC | 23:52 | |
| *** nelsnelson has quit IRC | 23:53 | |
| *** amotoki has joined #openstack-infra | 23:53 | |
| *** ChuckC_ has quit IRC | 23:54 | |
| *** patrickeast has quit IRC | 23:56 | |
| *** yamahata has quit IRC | 23:56 | |
| *** patrickeast has joined #openstack-infra | 23:57 | |
| *** ryanpetrello has joined #openstack-infra | 23:57 | |
| *** jaypipes has quit IRC | 23:57 | |
| *** todd_dsm has quit IRC | 23:57 | |
| *** amotoki has quit IRC | 23:58 | |
| *** marcoemorais has quit IRC | 23:58 | |
| clarkb | lifeless: how is it that you guys don't run out of space doing these builds? | 23:58 |
| clarkb | lifeless: you all just rock giant /'s all the time | 23:59 |
| *** marcoemorais has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
