| *** bswartz has joined #openstack-infra | 00:00 | |
| *** jklare has joined #openstack-infra | 00:00 | |
| *** zz_sabari is now known as sabari | 00:01 | |
| *** xyang1 has joined #openstack-infra | 00:01 | |
| *** MarkAtwood has joined #openstack-infra | 00:03 | |
| *** bswartz has quit IRC | 00:04 | |
| *** bswartz has joined #openstack-infra | 00:05 | |
| armax | mtreinish: ping | 00:06 |
|---|---|---|
| mtreinish | armax: pong | 00:08 |
| armax | mtreinish: thanks for eyeballing patch https://review.openstack.org/#/c/140851/ | 00:08 |
| armax | mtreinish: quick question though | 00:08 |
| armax | the file http://logs.openstack.org/51/140851/23/check//check-neutron-dsvm-functional/168d32e/logs/subunit_log.txt.gz | 00:08 |
| armax | looks good to me, what’s going on? can you tell? | 00:08 |
| mtreinish | oh, sure if you download it the file it's either apache, or the http clients assume it's text encoded | 00:09 |
| mtreinish | but it's really a binary so it'll screw things up | 00:10 |
| mtreinish | one sec, I'll find the d-g patch about it | 00:10 |
| *** amitgandhinz has joined #openstack-infra | 00:10 | |
| armax | mtreinish: I see | 00:10 |
| mtreinish | armax: http://git.openstack.org/cgit/openstack-infra/devstack-gate/commit/?id=f1838e89869c6e009c99253c2965132ea28ee30c | 00:11 |
| armax | mtreinish: gotcha now | 00:11 |
| armax | now I am with you | 00:11 |
| armax | mtreinish: thanks for bearing with me! | 00:11 |
| armax | mtreinish: I can see that the file is all garbled | 00:12 |
| armax | mtreinish: you told me yesteday too..but that slipped my mind! my bad | 00:12 |
| mtreinish | no worries | 00:12 |
| *** amitgandhinz has quit IRC | 00:15 | |
| *** shakamunyi has quit IRC | 00:15 | |
| clarkb | mtreinish: uh weshould've fixed that | 00:16 |
| clarkb | mtreinish: so if its still doing that we should figure out why | 00:16 |
| *** salv-orlando has joined #openstack-infra | 00:16 | |
| mtreinish | clarkb: armax has a patch adding it for the neutron functional jobs | 00:16 |
| mtreinish | it should be fixed for the tempest jobs | 00:17 |
| clarkb | mtreinish: it should be fixed for all jobs and that subunit log looks fine to me | 00:17 |
| *** dannywilson has quit IRC | 00:17 | |
| mtreinish | clarkb: it only becomes an issue when you download it | 00:17 |
| clarkb | mtreinish: thats what I did... | 00:18 |
| clarkb | wget | 00:18 |
| mtreinish | did you try to parse it | 00:18 |
| mtreinish | with one of the subunit tools? | 00:18 |
| clarkb | not yet I can though | 00:18 |
| mtreinish | yeah, that's where it screwed up. Like the subunit gearman workers would fail before we did the rename | 00:18 |
| clarkb | worked fine for me | 00:19 |
| clarkb | Ran 24 tests in 49.357s | 00:19 |
| *** palar has joined #openstack-infra | 00:19 | |
| mtreinish | hmm, weird. I know it used to fail with a .txt extension before | 00:19 |
| clarkb | from `testr load < filepath` | 00:19 |
| fungi | we added mod_mime_magic or something right? | 00:20 |
| *** barra204 has quit IRC | 00:20 | |
| clarkb | fungi: ya basically says that mime type is not test/plain | 00:20 |
| clarkb | and its binary data instead | 00:20 |
| fungi | just trying to remember how/where we fixed that on the logs vhost | 00:20 |
| clarkb | *text/plain | 00:20 |
| *** Ryan_Lane has quit IRC | 00:21 | |
| *** thedodd has quit IRC | 00:21 | |
| mtreinish | oh, it's a subunit v2 stream, before it was subunit v1 | 00:21 |
| *** salv-orlando has quit IRC | 00:22 | |
| *** bswartz has quit IRC | 00:22 | |
| clarkb | mtreinish: we did that because parsing v2 is faster than parsing v1 and v1 was really slow with neutrno | 00:22 |
| clarkb | mtreinish: and lifeless said v2 ould be default disk format in the near future | 00:22 |
| *** wenlock has quit IRC | 00:24 | |
| fungi | heh, neutrino | 00:24 |
| fungi | should totally be the new name for neutronclient | 00:25 |
| *** ryanpetrello has quit IRC | 00:25 | |
| *** salv-orlando has joined #openstack-infra | 00:25 | |
| flashgordon | clarkb: https://review.openstack.org/#/c/135768/6 can use a fresh review | 00:26 |
| flashgordon | as I fixed the commit message | 00:26 |
| clarkb | flashgordon: perfect much easier to grok now | 00:27 |
| clarkb | +2 | 00:27 |
| *** andreykurilin_ has quit IRC | 00:28 | |
| lifeless | clarkb: testr doesn't sort | 00:28 |
| lifeless | clarkb: it just prtitions | 00:28 |
| flashgordon | clarkb: thanks, I think another patch or two may have lost your +2 in the rebase | 00:28 |
| clarkb | lifeless: I thought you made sort order deterministic | 00:28 |
| *** salv-orlando has quit IRC | 00:29 | |
| clarkb | lifeless: because taking a subunit file from one host to another and running analze isoltion wouldn't work right otherwise | 00:29 |
| lifeless | in testtools.run yes within some limits | 00:29 |
| lifeless | no | 00:29 |
| lifeless | backend issue today | 00:29 |
| *** sweston_ has joined #openstack-infra | 00:29 | |
| *** sweston has quit IRC | 00:29 | |
| *** sweston_ is now known as sweston | 00:29 | |
| *** dims_ has joined #openstack-infra | 00:30 | |
| *** bswartz has joined #openstack-infra | 00:32 | |
| *** KurtMartin has joined #openstack-infra | 00:33 | |
| *** KurtMartin has quit IRC | 00:33 | |
| *** sweston_ has joined #openstack-infra | 00:33 | |
| *** ryanpetrello has joined #openstack-infra | 00:34 | |
| *** sweston has quit IRC | 00:35 | |
| *** sweston_ is now known as sweston | 00:35 | |
| *** salv-orlando has joined #openstack-infra | 00:36 | |
| *** Masahiro has joined #openstack-infra | 00:36 | |
| *** kmartin has quit IRC | 00:36 | |
| lifeless | clarkb: testr doesn't control sort order within a backend is the issue | 00:38 |
| clarkb | gotcha | 00:38 |
| clarkb | lifeless: basically backend says here is list and that may or may not be sorted in any particular order and testr won't change what it gets other than partitioning it | 00:39 |
| *** dimtruck is now known as zz_dimtruck | 00:39 | |
| *** salv-orlando has quit IRC | 00:40 | |
| *** Masahiro has quit IRC | 00:40 | |
| *** patrickeast has quit IRC | 00:43 | |
| *** ryanpetrello has quit IRC | 00:43 | |
| *** MarkAtwood has quit IRC | 00:46 | |
| *** ryanpetrello has joined #openstack-infra | 00:46 | |
| mestery | fungi: Thanks! Cloning and testing now, you rock sir! | 00:50 |
| *** mmaglana_ has quit IRC | 00:50 | |
| *** ChuckC has quit IRC | 00:51 | |
| *** david-lyle is now known as david-lyle_afk | 00:51 | |
| *** ryanpetrello has quit IRC | 00:51 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Added cron plugin to clean old access tokens. https://review.openstack.org/140536 | 00:52 |
| *** salv-orlando has joined #openstack-infra | 00:55 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT MERGE: testing aiopcpu with tempest full https://review.openstack.org/136504 | 00:55 |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Set custom cpu_model for live_migrate https://review.openstack.org/141530 | 00:55 |
| *** salv-orlando has quit IRC | 00:55 | |
| *** shashankhegde has quit IRC | 01:02 | |
| *** talluri has joined #openstack-infra | 01:03 | |
| *** talluri has quit IRC | 01:06 | |
| *** palar has quit IRC | 01:06 | |
| HenryG | armax: around? | 01:09 |
| armax | HenryG: yes, I am | 01:09 |
| HenryG | armax: Can vendor drivers for VPN/FW/LB also decompose? | 01:09 |
| armax | HenryG: dougwig and I thought that this could happen at a later stage | 01:10 |
| armax | HenryG: I think his spec captures this point, as in their community is still growing up and they may want to still be close together | 01:10 |
| *** ChuckC has joined #openstack-infra | 01:11 | |
| armax | HenryG: it’s like a marriage, in the early years lots of love, in the late years everyone sleeps in his/her own room | 01:11 |
| HenryG | armax: :) | 01:12 |
| *** ryanpetrello has joined #openstack-infra | 01:12 | |
| *** Ryan_Lane has joined #openstack-infra | 01:14 | |
| *** salv-orl_ has joined #openstack-infra | 01:21 | |
| *** salv-orl_ has quit IRC | 01:22 | |
| *** salv-orl_ has joined #openstack-infra | 01:22 | |
| *** gyee has quit IRC | 01:23 | |
| *** ryanpetrello has quit IRC | 01:24 | |
| dougwig | armax: too much truth there. :) | 01:27 |
| armax | dougwig: oh well, it’s been a long week | 01:27 |
| dougwig | Lol | 01:27 |
| *** salv-orl_ has quit IRC | 01:35 | |
| *** salv-orlando has joined #openstack-infra | 01:36 | |
| *** sabari is now known as zz_sabari | 01:37 | |
| *** shakamunyi has joined #openstack-infra | 01:39 | |
| *** barra204 has joined #openstack-infra | 01:39 | |
| *** dims_ has quit IRC | 01:43 | |
| *** dimsum__ has joined #openstack-infra | 01:43 | |
| *** shakamunyi has quit IRC | 01:44 | |
| *** dimsum__ has quit IRC | 01:48 | |
| *** dimsum__ has joined #openstack-infra | 01:48 | |
| *** barra204 has quit IRC | 01:49 | |
| *** zhiwei has joined #openstack-infra | 02:01 | |
| *** ryanpetrello has joined #openstack-infra | 02:02 | |
| lifeless | clarkb: nearly | 02:06 |
| lifeless | clarkb: testr says 'here is a list' | 02:06 |
| lifeless | clarkb: and backend runs in whatever order it wants | 02:06 |
| lifeless | clarkb: e.g. random, or sorted, or topologically sorted, or ... | 02:06 |
| lifeless | clarkb: testr needs to examine what comes back to determine isolation result validity and meaning | 02:07 |
| *** palar has joined #openstack-infra | 02:09 | |
| *** zhiwei has quit IRC | 02:13 | |
| *** ryanpetrello has quit IRC | 02:15 | |
| *** zhiwei has joined #openstack-infra | 02:19 | |
| *** ivar-laz_ has joined #openstack-infra | 02:24 | |
| *** Masahiro has joined #openstack-infra | 02:25 | |
| *** ivar-laz_ has quit IRC | 02:25 | |
| *** ivar-laz_ has joined #openstack-infra | 02:25 | |
| *** ivar-lazzaro has quit IRC | 02:28 | |
| *** Masahiro has quit IRC | 02:29 | |
| *** ryanpetrello has joined #openstack-infra | 02:30 | |
| *** ZZelle_ has quit IRC | 02:31 | |
| *** rkukura_ has joined #openstack-infra | 02:31 | |
| *** lttrl has quit IRC | 02:31 | |
| *** rkukura has quit IRC | 02:32 | |
| *** rkukura_ is now known as rkukura | 02:32 | |
| *** zhiwei has quit IRC | 02:33 | |
| *** zhiwei has joined #openstack-infra | 02:36 | |
| *** zhiwei has quit IRC | 02:37 | |
| *** palar has quit IRC | 02:42 | |
| *** ryanpetrello has quit IRC | 02:47 | |
| *** adalbas has quit IRC | 02:50 | |
| *** mmaglana has joined #openstack-infra | 02:52 | |
| *** mmaglana has quit IRC | 02:52 | |
| *** zz_sabari is now known as sabari | 02:52 | |
| *** Ryan_Lane has quit IRC | 02:56 | |
| *** stevemar has quit IRC | 02:58 | |
| *** talluri has joined #openstack-infra | 02:59 | |
| *** pc_m has quit IRC | 03:06 | |
| *** sweston has quit IRC | 03:07 | |
| *** ivar-laz_ has quit IRC | 03:08 | |
| *** sweston has joined #openstack-infra | 03:09 | |
| *** Ryan_Lane has joined #openstack-infra | 03:18 | |
| *** Ryan_Lane has quit IRC | 03:19 | |
| *** Ryan_Lane has joined #openstack-infra | 03:26 | |
| *** liusheng has quit IRC | 03:27 | |
| *** liusheng has joined #openstack-infra | 03:28 | |
| *** Ryan_Lane has quit IRC | 03:29 | |
| *** ildikov has joined #openstack-infra | 03:30 | |
| *** Ryan_Lane has joined #openstack-infra | 03:36 | |
| *** xyang1 has quit IRC | 03:40 | |
| *** Masahiro has joined #openstack-infra | 03:41 | |
| *** Ryan_Lane has quit IRC | 03:42 | |
| *** Ryan_Lane has joined #openstack-infra | 03:43 | |
| *** Masahiro has quit IRC | 03:45 | |
| *** Ryan_Lane has quit IRC | 03:51 | |
| *** boris-42 has quit IRC | 03:53 | |
| *** dimsum__ has quit IRC | 03:59 | |
| *** dimsum__ has joined #openstack-infra | 04:00 | |
| *** thedodd has joined #openstack-infra | 04:00 | |
| *** erikmwilson has quit IRC | 04:01 | |
| *** thedodd has quit IRC | 04:01 | |
| *** dimsum__ has quit IRC | 04:04 | |
| *** MarkAtwood has joined #openstack-infra | 04:05 | |
| *** mfink has quit IRC | 04:17 | |
| *** mfink has joined #openstack-infra | 04:17 | |
| *** dpaterson has joined #openstack-infra | 04:19 | |
| *** eharney has quit IRC | 04:20 | |
| *** sabari is now known as zz_sabari | 04:32 | |
| *** eharney has joined #openstack-infra | 04:33 | |
| *** talluri has quit IRC | 04:34 | |
| *** harlowja is now known as harlowja_away | 04:34 | |
| *** SumitNaiksatam has joined #openstack-infra | 04:46 | |
| *** marun has quit IRC | 04:46 | |
| *** sweston_ has joined #openstack-infra | 04:53 | |
| *** sweston has quit IRC | 04:54 | |
| *** sweston_ is now known as sweston | 04:54 | |
| *** hdd has quit IRC | 04:56 | |
| *** dpaterson has quit IRC | 05:23 | |
| *** Masahiro has joined #openstack-infra | 05:29 | |
| *** tjones2 has quit IRC | 05:31 | |
| *** Masahiro has quit IRC | 05:34 | |
| *** liusheng has quit IRC | 05:39 | |
| *** liusheng has joined #openstack-infra | 05:40 | |
| *** MarkAtwood has quit IRC | 05:43 | |
| *** bhunter71 has quit IRC | 05:44 | |
| *** liusheng has quit IRC | 05:48 | |
| *** hdd has joined #openstack-infra | 06:01 | |
| *** MarkAtwood has joined #openstack-infra | 06:03 | |
| *** nelsnelson has quit IRC | 06:20 | |
| *** zz_sabari is now known as sabari | 06:24 | |
| *** sabari is now known as zz_sabari | 06:26 | |
| *** achanda has joined #openstack-infra | 06:28 | |
| *** rkukura_ has joined #openstack-infra | 06:36 | |
| *** rkukura has quit IRC | 06:36 | |
| *** rkukura_ is now known as rkukura | 06:36 | |
| *** achanda has quit IRC | 06:40 | |
| *** achanda has joined #openstack-infra | 06:40 | |
| *** otter768 has quit IRC | 06:41 | |
| *** achanda_ has joined #openstack-infra | 06:43 | |
| *** achanda has quit IRC | 06:45 | |
| *** reed has quit IRC | 06:56 | |
| *** hdd has quit IRC | 07:06 | |
| *** nikil22 has joined #openstack-infra | 07:07 | |
| nikil22 | hi in devstack gate is it possible to generate localrc file form particular checking. Because in CI system everything works fine for me except its taking always the master branch and not the newly checked in code form gerrit system | 07:14 |
| *** Masahiro has joined #openstack-infra | 07:15 | |
| *** Masahiro has quit IRC | 07:19 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add --average option to sql2subunit cli https://review.openstack.org/132119 | 07:21 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add new db api methods for getting test data from runs https://review.openstack.org/141547 | 07:21 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Refactor sql2subunit to use get_tests_run_dicts_from_run_id https://review.openstack.org/141548 | 07:21 |
| *** enikanorov has joined #openstack-infra | 07:23 | |
| *** achanda_ has quit IRC | 07:24 | |
| *** enikanorov_ has quit IRC | 07:26 | |
| *** achanda has joined #openstack-infra | 07:28 | |
| *** melwitt has quit IRC | 07:29 | |
| *** nikil22 has quit IRC | 07:34 | |
| *** achanda has quit IRC | 07:36 | |
| *** nikil22 has joined #openstack-infra | 07:40 | |
| *** liusheng has joined #openstack-infra | 07:54 | |
| *** rkukura_ has joined #openstack-infra | 08:00 | |
| *** armax has quit IRC | 08:01 | |
| *** rkukura has quit IRC | 08:02 | |
| *** rkukura_ is now known as rkukura | 08:02 | |
| *** dimsum__ has joined #openstack-infra | 08:03 | |
| *** Longgeek has joined #openstack-infra | 08:04 | |
| nikil22 | hi in CI system it clone neutron repo exactly the checked in code. But in localrc file i don't see the repo url. So please let me know how to pull exactly the checked in code and run the tempest. ex: http://paste.openstack.org/show/150304/ | 08:05 |
| *** Longgeek has quit IRC | 08:07 | |
| *** Longgeek has joined #openstack-infra | 08:07 | |
| *** dimsum__ has quit IRC | 08:08 | |
| *** e0ne has joined #openstack-infra | 08:15 | |
| *** achanda has joined #openstack-infra | 08:20 | |
| *** mrmartin has joined #openstack-infra | 08:26 | |
| *** otter768 has joined #openstack-infra | 08:42 | |
| *** otter768 has quit IRC | 08:47 | |
| *** mrmartin has quit IRC | 08:57 | |
| *** rkukura_ has joined #openstack-infra | 09:03 | |
| *** MarkAtwood has quit IRC | 09:03 | |
| *** Masahiro has joined #openstack-infra | 09:04 | |
| *** rkukura has quit IRC | 09:06 | |
| *** rkukura_ is now known as rkukura | 09:06 | |
| *** e0ne has quit IRC | 09:07 | |
| *** HeOS has quit IRC | 09:08 | |
| *** zz_avozza is now known as avozza | 09:09 | |
| *** Masahiro has quit IRC | 09:09 | |
| *** koolhead17 has joined #openstack-infra | 09:09 | |
| *** koolhead17 has joined #openstack-infra | 09:09 | |
| *** liusheng has quit IRC | 09:11 | |
| *** liusheng has joined #openstack-infra | 09:12 | |
| *** MaxV has joined #openstack-infra | 09:24 | |
| *** Guest77493 has joined #openstack-infra | 09:25 | |
| *** liusheng has quit IRC | 09:29 | |
| *** liusheng has joined #openstack-infra | 09:30 | |
| *** mrmartin has joined #openstack-infra | 09:47 | |
| *** fandi has quit IRC | 09:48 | |
| *** liusheng has quit IRC | 09:49 | |
| *** liusheng has joined #openstack-infra | 09:49 | |
| *** hashar has joined #openstack-infra | 09:51 | |
| nikil22 | hi in devstack-gate how to specify the reop/branch/commit ID. I get this Information in jenkins but not sure how to make job to pull the exact checked in version and run the devstack | 09:52 |
| *** hashar has quit IRC | 10:03 | |
| *** achanda has quit IRC | 10:06 | |
| *** sweston has quit IRC | 10:06 | |
| *** tnovacik has joined #openstack-infra | 10:07 | |
| *** Guest77493 has quit IRC | 10:08 | |
| *** fandi has joined #openstack-infra | 10:14 | |
| *** MaxV has quit IRC | 10:23 | |
| *** mrmartin has quit IRC | 10:32 | |
| *** enikanorov has quit IRC | 10:36 | |
| *** zz_sabari is now known as sabari | 10:37 | |
| *** otter768 has joined #openstack-infra | 10:43 | |
| *** koolhead17 has quit IRC | 10:47 | |
| *** otter768 has quit IRC | 10:47 | |
| *** Masahiro has joined #openstack-infra | 10:53 | |
| *** Masahiro has quit IRC | 10:57 | |
| *** ZZelle has quit IRC | 10:57 | |
| *** ZZelle has joined #openstack-infra | 10:58 | |
| *** asselin has quit IRC | 11:16 | |
| *** tnovacik has quit IRC | 11:17 | |
| *** MaxV has joined #openstack-infra | 11:24 | |
| *** sabari is now known as zz_sabari | 11:41 | |
| *** salv-orlando has quit IRC | 11:44 | |
| *** avozza is now known as zz_avozza | 11:46 | |
| *** _nadya_ has joined #openstack-infra | 11:46 | |
| *** tnovacik has joined #openstack-infra | 11:48 | |
| *** boris-42 has joined #openstack-infra | 11:50 | |
| *** pblaho has joined #openstack-infra | 11:54 | |
| *** andreykurilin_ has joined #openstack-infra | 11:55 | |
| *** MaxV has quit IRC | 11:58 | |
| *** pblaho has quit IRC | 12:00 | |
| *** achanda has joined #openstack-infra | 12:01 | |
| *** amotoki has joined #openstack-infra | 12:01 | |
| *** salv-orlando has joined #openstack-infra | 12:02 | |
| *** MaxV has joined #openstack-infra | 12:02 | |
| *** dimsum__ has joined #openstack-infra | 12:06 | |
| *** smoser has quit IRC | 12:07 | |
| *** MaxV has quit IRC | 12:08 | |
| *** dimsum__ has quit IRC | 12:10 | |
| *** skata_ has quit IRC | 12:11 | |
| *** ociuhandu has quit IRC | 12:16 | |
| *** _nadya_ has quit IRC | 12:16 | |
| *** fandi_ has joined #openstack-infra | 12:18 | |
| *** fandi has quit IRC | 12:18 | |
| *** Longgeek has quit IRC | 12:21 | |
| *** Longgeek has joined #openstack-infra | 12:22 | |
| *** Longgeek_ has joined #openstack-infra | 12:25 | |
| *** lttrl has joined #openstack-infra | 12:25 | |
| *** Longgeek has quit IRC | 12:28 | |
| *** zz_avozza is now known as avozza | 12:36 | |
| *** amotoki has quit IRC | 12:37 | |
| *** Masahiro has joined #openstack-infra | 12:41 | |
| *** otter768 has joined #openstack-infra | 12:44 | |
| *** Masahiro has quit IRC | 12:46 | |
| *** rkukura has quit IRC | 12:47 | |
| *** otter768 has quit IRC | 12:49 | |
| *** rkukura has joined #openstack-infra | 12:49 | |
| *** amotoki has joined #openstack-infra | 13:02 | |
| *** lttrl has quit IRC | 13:06 | |
| *** mjturek has quit IRC | 13:12 | |
| *** e0ne has joined #openstack-infra | 13:19 | |
| *** Longgeek_ has quit IRC | 13:20 | |
| *** Longgeek has joined #openstack-infra | 13:21 | |
| *** ryanpetrello has joined #openstack-infra | 13:21 | |
| *** Longgeek_ has joined #openstack-infra | 13:22 | |
| *** ZZelle_ has joined #openstack-infra | 13:24 | |
| *** Longgeek has quit IRC | 13:25 | |
| *** ryanpetrello has quit IRC | 13:28 | |
| *** MaxV has joined #openstack-infra | 13:31 | |
| *** andreykurilin_ has quit IRC | 13:33 | |
| *** avozza is now known as zz_avozza | 13:35 | |
| *** lttrl has joined #openstack-infra | 13:52 | |
| *** salv-orlando has quit IRC | 14:02 | |
| *** boris-42 has quit IRC | 14:03 | |
| *** amotoki has quit IRC | 14:03 | |
| *** Longgeek_ has quit IRC | 14:05 | |
| *** MaxV has quit IRC | 14:05 | |
| *** Longgeek has joined #openstack-infra | 14:06 | |
| *** mrmartin has joined #openstack-infra | 14:06 | |
| *** MaxV has joined #openstack-infra | 14:07 | |
| *** mrmartin has quit IRC | 14:08 | |
| *** Longgeek_ has joined #openstack-infra | 14:10 | |
| *** achanda has quit IRC | 14:10 | |
| *** amotoki has joined #openstack-infra | 14:12 | |
| *** fandi_ has quit IRC | 14:13 | |
| *** lttrl has quit IRC | 14:13 | |
| *** achanda has joined #openstack-infra | 14:13 | |
| *** Longgeek has quit IRC | 14:13 | |
| *** fandi has joined #openstack-infra | 14:13 | |
| *** fandi has quit IRC | 14:14 | |
| *** fandi has joined #openstack-infra | 14:15 | |
| *** achanda has quit IRC | 14:15 | |
| *** achanda has joined #openstack-infra | 14:16 | |
| *** subscope has joined #openstack-infra | 14:19 | |
| *** achanda has quit IRC | 14:20 | |
| *** fandi has quit IRC | 14:21 | |
| *** fandi has joined #openstack-infra | 14:21 | |
| *** koolhead17 has joined #openstack-infra | 14:24 | |
| *** zz_avozza is now known as avozza | 14:26 | |
| *** Longgeek_ has quit IRC | 14:27 | |
| *** Masahiro has joined #openstack-infra | 14:30 | |
| *** salv-orlando has joined #openstack-infra | 14:32 | |
| *** Longgeek has joined #openstack-infra | 14:33 | |
| *** Longgeek has quit IRC | 14:34 | |
| *** Masahiro has quit IRC | 14:35 | |
| *** avozza is now known as zz_avozza | 14:36 | |
| *** salv-orlando has quit IRC | 14:38 | |
| *** _nadya_ has joined #openstack-infra | 14:39 | |
| *** HeOS has joined #openstack-infra | 14:40 | |
| *** Longgeek has joined #openstack-infra | 14:41 | |
| *** otter768 has joined #openstack-infra | 14:45 | |
| *** koolhead17 has quit IRC | 14:46 | |
| *** salv-orlando has joined #openstack-infra | 14:47 | |
| *** otter768 has quit IRC | 14:49 | |
| *** achanda has joined #openstack-infra | 14:50 | |
| *** marcusvrn has quit IRC | 14:52 | |
| *** marcusvrn has joined #openstack-infra | 14:54 | |
| *** marcusvrn has quit IRC | 14:58 | |
| *** marcusvrn has joined #openstack-infra | 15:00 | |
| *** dimsum__ has joined #openstack-infra | 15:08 | |
| *** liusheng has quit IRC | 15:10 | |
| *** dimsum__ has quit IRC | 15:13 | |
| *** liusheng has joined #openstack-infra | 15:14 | |
| *** koolhead17 has joined #openstack-infra | 15:22 | |
| *** zz_avozza is now known as avozza | 15:27 | |
| *** yfried_ has joined #openstack-infra | 15:28 | |
| *** _nadya_ has quit IRC | 15:36 | |
| *** pc_m has joined #openstack-infra | 15:46 | |
| *** sweston has joined #openstack-infra | 15:49 | |
| *** carl_baldwin has joined #openstack-infra | 15:52 | |
| *** subscope has quit IRC | 15:53 | |
| *** ChanServ has quit IRC | 15:55 | |
| *** sweston has left #openstack-infra | 15:55 | |
| *** salv-orlando has quit IRC | 15:56 | |
| *** nikil22 has quit IRC | 15:57 | |
| *** koolhead17 has quit IRC | 16:00 | |
| *** _nadya_ has joined #openstack-infra | 16:07 | |
| *** carl_baldwin has quit IRC | 16:09 | |
| *** MaxV has quit IRC | 16:11 | |
| *** _nadya_ has quit IRC | 16:15 | |
| *** Masahiro has joined #openstack-infra | 16:19 | |
| *** MaxV has joined #openstack-infra | 16:22 | |
| *** paul-- has quit IRC | 16:22 | |
| *** ChanServ has joined #openstack-infra | 16:23 | |
| *** sendak.freenode.net sets mode: +o ChanServ | 16:23 | |
| *** Masahiro has quit IRC | 16:23 | |
| *** andreykurilin_ has joined #openstack-infra | 16:24 | |
| *** paul-- has joined #openstack-infra | 16:27 | |
| *** marcusvrn has quit IRC | 16:33 | |
| *** marcusvrn has joined #openstack-infra | 16:33 | |
| *** yfried_ has quit IRC | 16:41 | |
| *** boris-42 has joined #openstack-infra | 16:42 | |
| *** harlowja_at_home has joined #openstack-infra | 16:42 | |
| *** marcusvrn has quit IRC | 16:44 | |
| *** e0ne has quit IRC | 16:45 | |
| *** otter768 has joined #openstack-infra | 16:45 | |
| *** marcusvrn has joined #openstack-infra | 16:46 | |
| *** MaxV has quit IRC | 16:49 | |
| *** otter768 has quit IRC | 16:50 | |
| *** yfried_ has joined #openstack-infra | 16:53 | |
| fungi | ugh... freenode's upgraded to an atheme release which drops dh-blowfish support for sasl, yet weechat is still on the fence about adding ecdsa-nist256p-challenge | 16:53 |
| *** harlowja_at_home has quit IRC | 16:54 | |
| fungi | so basically, the options are between plain auth and plain auth | 16:55 |
| * fungi grumbles and goes back to having a saturday | 16:55 | |
| *** salv-orlando has joined #openstack-infra | 16:57 | |
| *** armax has joined #openstack-infra | 16:59 | |
| *** salv-orlando has quit IRC | 17:01 | |
| *** salv-orlando has joined #openstack-infra | 17:01 | |
| *** yfried_ has quit IRC | 17:14 | |
| *** HeOS has quit IRC | 17:15 | |
| *** yfried_ has joined #openstack-infra | 17:17 | |
| *** dimsum__ has joined #openstack-infra | 17:17 | |
| *** otter768 has joined #openstack-infra | 17:17 | |
| *** erlon has quit IRC | 17:19 | |
| *** achanda has quit IRC | 17:20 | |
| *** MaxV has joined #openstack-infra | 17:20 | |
| *** achanda has joined #openstack-infra | 17:21 | |
| *** amitgandhinz has joined #openstack-infra | 17:21 | |
| *** achanda has quit IRC | 17:23 | |
| *** achanda has joined #openstack-infra | 17:23 | |
| *** MaxV has quit IRC | 17:24 | |
| *** armax has quit IRC | 17:29 | |
| *** armax has joined #openstack-infra | 17:38 | |
| *** nikil22 has joined #openstack-infra | 17:41 | |
| nikil22 | Hi In CI system how to use the exact branch/commit ID in the JOB. i get these details as a variable in jenkins. But how can i use it in the devstack-gate so that it will clone that exact changes and run the test | 17:41 |
| *** _nadya_ has joined #openstack-infra | 17:42 | |
| *** fifieldt_ has quit IRC | 17:44 | |
| *** fifieldt_ has joined #openstack-infra | 17:45 | |
| *** amitgandhinz has quit IRC | 17:46 | |
| *** shashankhegde has joined #openstack-infra | 17:46 | |
| *** amitgandhinz has joined #openstack-infra | 17:46 | |
| *** tjones1 has joined #openstack-infra | 17:49 | |
| fungi | nikil22: we use zuul to calculate the git refs and supply them to jenkins via gearman | 17:50 |
| fungi | nikil22: but i think different people do it different ways | 17:51 |
| nikil22 | fungi: same thing happens . Actually i get it via gearman. But how exactly in the job i should use it | 17:51 |
| *** _nadya_ has quit IRC | 17:51 | |
| *** shakamunyi has joined #openstack-infra | 17:51 | |
| nikil22 | fungi: Because when the example job like dsvm-tempest-full will download only the master branch and run test in it. | 17:52 |
| *** barra204 has joined #openstack-infra | 17:52 | |
| fungi | nikil22: hopefully some of the people running third-party ci systems will speak up. i can explain how the openstack infrastructure ci integrates those pieces, but that may or may not be applicable to your use case | 17:52 |
| fungi | er, well, i can explain later. at the moment i have to go run weekend errands | 17:52 |
| fungi | nikil22: devstack-gate determines the branches and refs to use based on what zuul determines from gerrit (the change proposed, the branch it's proposed against, whether that's a branch present on other projects being integrated, et cetera) | 17:53 |
| * fungi will be back later | 17:53 | |
| *** tjones1 has quit IRC | 17:54 | |
| *** avozza is now known as zz_avozza | 17:55 | |
| nikil22 | fungi: thanks i will also look into more about the devstack-gate. Still now i dont see any variable to set the reference only branch and like neutron to enable or not i see. any way thanks i will dig more deeper . | 17:55 |
| *** liusheng has quit IRC | 18:00 | |
| *** liusheng has joined #openstack-infra | 18:01 | |
| *** barra204 has quit IRC | 18:03 | |
| *** cody-somerville has quit IRC | 18:03 | |
| *** dimsum__ has quit IRC | 18:04 | |
| *** dimsum__ has joined #openstack-infra | 18:05 | |
| *** Masahiro has joined #openstack-infra | 18:08 | |
| *** dimsum__ has quit IRC | 18:09 | |
| *** dimsum__ has joined #openstack-infra | 18:09 | |
| *** Masahiro has quit IRC | 18:12 | |
| *** AJaeger has joined #openstack-infra | 18:13 | |
| AJaeger | fungi, jeblair, anteaya: Could you have a look at https://review.openstack.org/#/c/141171/7/doc/source/devref/contribute.rst and tell me whether I'm overreacting, please? | 18:13 |
| *** barra204 has joined #openstack-infra | 18:17 | |
| *** juris has joined #openstack-infra | 18:17 | |
| *** juris has left #openstack-infra | 18:18 | |
| *** melwitt has joined #openstack-infra | 18:24 | |
| *** e0ne has joined #openstack-infra | 18:25 | |
| *** armax has quit IRC | 18:27 | |
| *** andreykurilin_ has quit IRC | 18:39 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Run an openstackweb dev site https://review.openstack.org/141571 | 18:42 |
| *** amitgandhinz has quit IRC | 18:42 | |
| clarkb | that ^ has been WIP'd it is untested, but I figured we needed to start somewhere and I think that gives some initial shape to the thing | 18:44 |
| clarkb | if you grok php, silverstripe, openstackweb, apache, whatever please do push new patchsets. It will help a bunch | 18:45 |
| ruhe | something's strange with devstack jobs. here is paste from devstacklog http://paste.openstack.org/raw/150391/ | 18:46 |
| ruhe | i see it in various reviews | 18:47 |
| clarkb | ruhe: can you please link to specific job logs? we already host all of them | 18:47 |
| clarkb | no need to use paste for that | 18:47 |
| *** achanda has quit IRC | 18:47 | |
| ruhe | clarkb: here is one of them http://logs.openstack.org/97/114997/2/check//check-tempest-dsvm-full/2c5826c/logs/ | 18:47 |
| ruhe | i don't even understand how can it fail with such error if swift doesn't have oslo.db and sqlalchemy in requirements.txt | 18:48 |
| clarkb | it is also odd that the version it complains about seems to match the allowed versions | 18:50 |
| clarkb | ruhe: the error is hppening via paste. Is there not something in the paste config that does use oslo.db? (iirc it is ceilometer but would have to check) | 18:50 |
| *** leakypipes has quit IRC | 18:50 | |
| clarkb | ya ceilometer is in the pipeline listed in proxy-server.conf | 18:52 |
| *** zz_sabari has quit IRC | 18:52 | |
| AJaeger | clarkb, ruhe: It says "SQLAlchemy<=0.8.99" - and 0.9.8 is isntalled | 18:55 |
| clarkb | its odd too that https://git.openstack.org/cgit/openstack/requirements/commit/?id=b52f1e8fd9a4ad5b6bb61ffbd59124159b55cd4e is what global requirements says | 18:55 |
| clarkb | AJaeger: ya but it also allows <=0.9.99 | 18:55 |
| AJaeger | clarkb: who is the logic for this? "<= 0.8.99 and <= 0.9.99"? So, that gives "<= 0.8.99", doesn't it? | 18:56 |
| clarkb | AJaeger: I don't think there is any logic... | 18:57 |
| AJaeger | ;( | 18:57 |
| clarkb | AJaeger: my understanding is they want to say 0.8 or 0.9 are ok | 18:57 |
| clarkb | AJaeger: which should be >=0.8.4,<0.10 | 18:58 |
| *** salv-orlando has quit IRC | 18:58 | |
| AJaeger | yeah... | 18:58 |
| clarkb | BUT old versions of pip will install betas and alphas if you do that hence the weirdness | 18:58 |
| *** zz_avozza is now known as avozza | 18:58 | |
| AJaeger | requirement repo just has "SQLAlchemy>=0.9.7,<=0.9.99" | 18:58 |
| clarkb | AJaeger: ya thats a nw update from this week, but we test against release versions of oslo.db | 18:59 |
| clarkb | so oslo.db still has the old version. If that is a problem I would've expected tests to fail on global requirements chnage | 18:59 |
| clarkb | it should've failed at https://review.openstack.org/#/c/139880/ if that was a problem | 18:59 |
| *** shashankhegde has quit IRC | 19:00 | |
| AJaeger | understood, thanks | 19:00 |
| *** zz_sabari has joined #openstack-infra | 19:01 | |
| *** _nadya_ has joined #openstack-infra | 19:02 | |
| clarkb | I wonder | 19:03 |
| *** otter768 has quit IRC | 19:03 | |
| clarkb | no 0.9.8 was used according to pip freeze | 19:03 |
| clarkb | er was used when the new version spec got in | 19:03 |
| clarkb | and ceilometer was in the proxy-server pipeline then too | 19:04 |
| *** otter768 has joined #openstack-infra | 19:04 | |
| clarkb | did we just do an oslo.db release? | 19:04 |
| clarkb | nope | 19:05 |
| social | fungi: thanks a _lot_ | 19:05 |
| *** _nadya_ has quit IRC | 19:05 | |
| ruhe | i haven't found any new releases or related changes in these projects yet | 19:05 |
| *** HeOS has joined #openstack-infra | 19:06 | |
| clarkb | guess what released today? | 19:07 |
| clarkb | setuptools! | 19:07 |
| clarkb | so pretty sure its newer setuptools (where pkg_resources comes from) braking us | 19:07 |
| ruhe | :) | 19:07 |
| AJaeger | releases at weekends ;( | 19:07 |
| dstufft | it was released on a weekend because that's when the volunteer who maanges setuptools has time | 19:08 |
| * clarkb looks up setuptools bug tracker | 19:08 | |
| clarkb | dstufft: https://bitbucket.org/pypa/setuptools/issues that the one? | 19:09 |
| dstufft | yea | 19:09 |
| dstufft | that old SQLAlchemy qualifer isn't valid in PEP 440 | 19:09 |
| *** cody-somerville has joined #openstack-infra | 19:09 | |
| clarkb | dstufft: ya but I think over in -dev that one is valid | 19:09 |
| AJaeger | dstufft: yeah, understandable - it's just that I noticed too often breakage with release at weekends. Not sure whether it's my perception filter or whether there's a real corrolation | 19:10 |
| dstufft | clarkb: uh, let me look at that one | 19:10 |
| clarkb | dstufft: ERROR: openstackclient.shell Exception raised: python-neutronclient 2.3.9.40.g9ed73c0 is installed but python-neutronclient<3,>=2.3.6 is required by [] | 19:10 |
| dstufft | clarkb: fwiw the specifiers are written by me | 19:10 |
| clarkb | dstufft: isn't that valid? | 19:10 |
| dstufft | clarkb: oh lol I know what the problem is | 19:10 |
| clarkb | dstufft: shoudl you file the bug then? because I have no idea | 19:10 |
| clarkb | I was just going to paste ^ and sa fix it :) | 19:10 |
| dstufft | it's because you're mixing a non PEP 440 version with a PEP 440 specifier | 19:11 |
| clarkb | I'm not sure I follow | 19:11 |
| clarkb | aren'tboth of those 440? | 19:11 |
| *** otter768 has quit IRC | 19:11 | |
| dstufft | so let me explain | 19:11 |
| dstufft | PEP 440 defines an order for versions that match PEP 440 right? | 19:11 |
| dstufft | a sort order | 19:11 |
| dstufft | however there are cases where people had versions that _didnt_ match PEP 440, so we needed to do something to make it possible to sort them | 19:12 |
| dstufft | so what we did is essentially sort all versions which are not PEP 440 compatible as "less than" any PEP 440 compat version | 19:12 |
| dstufft | (and within that, use the old setuptools algorithm) | 19:12 |
| *** pcrews has quit IRC | 19:12 | |
| clarkb | tl;dr all of python is going to be broken for a while | 19:13 |
| dstufft | so your specifier has >=2.3.6, but what that really is saying is >= (1, 2, 3, 6), but your version is (-1, 2, 3, 9, 40, "g9ed73c0") | 19:13 |
| clarkb | I thought 2.3.9.40.g9ed73c0 was 440 compliant | 19:14 |
| *** e0ne has quit IRC | 19:14 | |
| dstufft | no | 19:14 |
| *** otter768 has joined #openstack-infra | 19:14 | |
| dstufft | it's the .g9ed73c0 that makes it not | 19:14 |
| dstufft | there's no sane way to sort that | 19:14 |
| clarkb | yes there is s/g9ed73c0// | 19:15 |
| dstufft | if you change that to 2.3.9.40+g9ed73c0 it'll work, because the+ signifiies a "local version" which has no semantics | 19:15 |
| dstufft | clarkb: we can't just ignore information | 19:15 |
| AJaeger | What fun ;( | 19:15 |
| clarkb | dstufft: apparently thats what the + does... | 19:15 |
| clarkb | anyways I am going to go be angry over there -> now | 19:15 |
| dstufft | clarkb: the + doesn't ignore it, it still has a defined sort order, it sorts by lower() and then string sorting, but it's special because things like PyPI won't allow you to upload it and stuff | 19:16 |
| clarkb | dstufft: so ignoring neutronclient thing | 19:16 |
| clarkb | dstufft: why is the sqlalchemy one not 440 compliant | 19:16 |
| clarkb | best I can tell all of the sqlalchemy version stuff is compliant | 19:17 |
| dstufft | clarkb: I slightly mispoke, the problem isn't that it's not compatible (the SQLALchemy one), the problem is it relies on an inconsistent use of commas | 19:17 |
| dstufft | SQLAlchemy>=0.8.4,<=0.8.99,>=0.9.7,<=0.9.99 | 19:18 |
| dstufft | if you treat the , as an AND, then it matches nothing, if you treat is as an OR, then it matches everything | 19:18 |
| clarkb | dstufft: so two changes in behavior then? are they both related to 400? | 19:18 |
| clarkb | er 440? | 19:18 |
| dstufft | you have to treat commas as both an OR and an AND in order for that specifier to make sense | 19:18 |
| dstufft | clarkb: yea | 19:18 |
| clarkb | yes I agree it makes no logical sense, but it worked in previous setuptools | 19:19 |
| clarkb | so how are you supposed to supply information about where a version comes from without making it sortable? | 19:19 |
| dstufft | clarkb: well lots of things worked in previous setuptools, part of the effort we're doing with packaging is standardizing and documenting, and as part of that we're also breaking behaviors that don't make sense | 19:19 |
| clarkb | sure | 19:20 |
| clarkb | except I think this just liteally broke the python world | 19:20 |
| clarkb | bceause evne if I fix my thing half of pypi will be broken | 19:20 |
| dstufft | no | 19:20 |
| dstufft | we tested PEP 440 against PyPI | 19:20 |
| clarkb | but apparently not setuptools 8.0 | 19:20 |
| dstufft | we can successfully parse 98.12% as PEP 440 compliant versions | 19:20 |
| clarkb | otherwise oslo.db would've flagged an error | 19:20 |
| dstufft | I didn't say we achieved 100% compatible | 19:21 |
| clarkb | dstufft: I think you are saying the versions on pypi are compatible. what I am saying is the versions people use in their requirements lists may not be | 19:21 |
| clarkb | dstufft: or am I misunderstanding what you mean by testing 440 against pypi | 19:21 |
| dstufft | clarkb: sorry that was just the number I copy/pasted from the PEP | 19:22 |
| dstufft | we did more testing than that | 19:22 |
| dstufft | in all cases we were within a few percent of 100% | 19:22 |
| dstufft | openstack's requirement for SQLAlchemy is an outlier not a common case | 19:22 |
| dstufft | I've also been telling y'all it's gonna break in the future for months now :( | 19:23 |
| clarkb | dstufft: I don't recall that. I do recall being told to read pep 440 | 19:24 |
| clarkb | which I did and I misunderstood our versions as being compatible | 19:24 |
| clarkb | in any case I need to go weekend now | 19:24 |
| AJaeger | should we cap setuptools for now? Or any other suggestion as next steop? | 19:25 |
| clarkb | AJaeger: that is probably the most direct immediate thing to do | 19:25 |
| AJaeger | to < 8.0? | 19:26 |
| dstufft | you probably want to cap pip to < 6 | 19:26 |
| dstufft | as well | 19:26 |
| dstufft | because pip 6 includes PEP 440 | 19:27 |
| dstufft | and will be coming out next week | 19:27 |
| AJaeger | setuptools is not in global-requirements ;( | 19:27 |
| clarkb | AJaeger: no beacuse it is expected to be there | 19:27 |
| clarkb | AJaeger: we have to pin it in our images | 19:27 |
| clarkb | setuptools is special | 19:27 |
| AJaeger | clarkb: I see... | 19:27 |
| AJaeger | not sure what to change for that, so I let you or others do that and go back to my weekend as well ... | 19:29 |
| clarkb | reeading pbr version.py seems to indicate 440 was considered too | 19:35 |
| clarkb | so clearly we missed something reading 440 | 19:35 |
| *** otter768 has quit IRC | 19:38 | |
| clarkb | oh thats unreleased so that explains that (but the unreleased code would be broken too) | 19:39 |
| *** Longgeek has quit IRC | 19:47 | |
| * mordred is unhappy | 19:50 | |
| *** shashankhegde has joined #openstack-infra | 19:52 | |
| *** Longgeek has joined #openstack-infra | 19:52 | |
| ruhe | setting a cap on pip to < 6 seems complicated because of get-pip.py, while something like 'pip install -U "setuptools<8.0"' in install_puppet.sh should do the job | 19:52 |
| mordred | ruhe: yeah - we're probably going to need to pin in devstack as well just to be sure | 19:53 |
| dstufft | you can do ``pip install -U pip<6.0`` too, even after you do get-pip.py | 19:53 |
| *** amitgandhinz has joined #openstack-infra | 19:53 | |
| mordred | dstufft: the version fallback logic is insane | 19:53 |
| mordred | it's not what anyone is ever going to expect | 19:54 |
| dstufft | mordred: the alternative is an error | 19:54 |
| mordred | only if you make an arbitrary set of rules that aren't logical and then stick to them with no regard to common sense | 19:55 |
| mordred | I get taht yuo can't sort g1324234 | 19:55 |
| mordred | but making 2.3.9.gfoo < 2.3.6 is the least obvious failure mode possible | 19:55 |
| mordred | when CLEARLY the 9 is greated than the 6 | 19:55 |
| *** mwagner_lap has quit IRC | 19:56 | |
| *** Longgeek has quit IRC | 19:56 | |
| *** Masahiro has joined #openstack-infra | 19:57 | |
| dstufft | mordred: very few of the versions on PyPI had a pattern like that | 19:57 |
| mordred | sure - and we don't _release_ ours - but that doesn't mean that people with things not on pypi aren't going to be screwed | 19:57 |
| mordred | and I'd say all over the place | 19:57 |
| mordred | because lotgs of people who do not publish things to pypi also don't read things like PEP440 | 19:57 |
| mordred | imagine the crazy pain this is going to cause for people with python-based corporate apps that all of a sudden arent' going to install because they weren't pure enough in their thinking brain | 19:58 |
| mordred | I'm just upset because not I'm going to spend the next however many hours or days scrambling to unbreak us | 19:59 |
| mordred | also, do we have the field in the new metadata system to put the git sha information yet? beacuse I've been saying for forever that we'd be more than happy to move our extra info to that once it exists | 20:00 |
| clarkb | mordred: I think +g for local info | 20:00 |
| mordred | but it seems that we've released the enforcement without having released the updates to metadata 2.0 | 20:00 |
| clarkb | and still shove it into the version | 20:01 |
| *** Masahiro has quit IRC | 20:01 | |
| mordred | great. | 20:01 |
| * mordred fumes more | 20:01 | |
| dstufft | mordred: I've been asking people to help test this change for months, I've been telling people this might break things for month. I've used every avenue I've had at my disposal to try and get information from people that had things that wern't on PyPI | 20:01 |
| dstufft | If some corporate app lives in a vacuum I can't really do a lot about that | 20:02 |
| clarkb | so I think wheer the disconnect was for me at least is that setuptools would enforce pep440. And do so without a WARNING THIS VERSION IS GOING TO BREAK LATER period | 20:02 |
| mordred | right | 20:02 |
| mordred | pip enforcing this makes perfect sense | 20:03 |
| mordred | and we were totally on board with that that meant | 20:03 |
| mordred | setuptools enforcing it in this way is a complete and total surprise | 20:03 |
| dstufft | I'm not sure what breakage you think would not have happened if setuptools hadn't done it as well | 20:03 |
| mordred | we install a version fromgit | 20:03 |
| dstufft | how do you install it | 20:04 |
| mordred | then we install a thing that depends on that thing which declares a thing | 20:04 |
| mordred | pip install -e . | 20:04 |
| mordred | in the git repo | 20:04 |
| mordred | THEN | 20:04 |
| mordred | in a different git repo | 20:04 |
| mordred | we install another thing that depends on the first thing which we've already installed | 20:04 |
| mordred | except that install breaks | 20:04 |
| dstufft | and how do you install that | 20:04 |
| mordred | because now the thing we installed sorts too low | 20:04 |
| mordred | pip install . | 20:04 |
| dstufft | so your install still would have broken | 20:04 |
| mordred | awesome | 20:04 |
| dstufft | if pip enforced it ands etuptools didn't | 20:04 |
| clarkb | no not quite | 20:04 |
| mordred | I'm so happy about the value this has brought me | 20:05 |
| clarkb | I mean it would still be funky | 20:05 |
| clarkb | but pip would install a "newer" version instead | 20:05 |
| dstufft | pip would have uninstalled the git version and installed the latest version from PyPI it could find | 20:05 |
| clarkb | because pip would say you need X let me go get that | 20:05 |
| dstufft | which is probably not what you wanted | 20:05 |
| clarkb | and things would still work | 20:05 |
| *** melwitt has quit IRC | 20:06 | |
| clarkb | but that is where the THIS VERISON IS GOING TO BREAK warning comes from | 20:06 |
| dstufft | clarkb: the problem with that is, in our ability to determine, what PEP 440 did is for _most_ people, it made the versions and the specifiers they were using work more like what they thought it was going to work like | 20:07 |
| dstufft | so "this is going to break" isn't exactly correct | 20:08 |
| dstufft | because for most people it's "in the future this is going to be doing what you thought you were doing" | 20:08 |
| clarkb | well except for the sort order being used | 20:08 |
| clarkb | you could emit a warning any time you had to fall back on funky sort order | 20:08 |
| *** amitgandhinz has quit IRC | 20:08 | |
| clarkb | or would | 20:09 |
| *** dimsum__ has quit IRC | 20:09 | |
| clarkb | because parsefailure -> sort first is likely unexpected in all situations? | 20:09 |
| *** salv-orlando has joined #openstack-infra | 20:09 | |
| dstufft | if you think a warning about the funky sort order would make things better I can make a PR for that right now and see if jason can publish a quick 8.0.1 with it | 20:10 |
| clarkb | I think it would've at least made debugging easier. If you weren't in channel there is no way I would've figured that out | 20:10 |
| clarkb | (I think that warning should've come in a release before enforcement but I guess that ship has sailed) | 20:10 |
| * dstufft tries to think how best to add a warning | 20:12 | |
| *** armax has joined #openstack-infra | 20:13 | |
| *** salv-orlando has quit IRC | 20:14 | |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/system-config: Pin version of setuptools https://review.openstack.org/141577 | 20:15 |
| fungi | i don't get why g1324234 is unsortable. isn't it just an alphanumeric string? posix has a well defined sort order for those since... well... posix? | 20:18 |
| fungi | though i am hoping people with "python-based corporate apps" not reading peps and following release notes also aren't automatically upgrading to most recent setuptools, et cetera | 20:19 |
| *** ryanpetrello has joined #openstack-infra | 20:19 | |
| dstufft | fungi: logically sortable, obviously any data can have a sort order defined for it, but like, is the version "dog" newer or older than the version "cat" | 20:20 |
| ruhe | https://review.openstack.org/#/c/141577/ https://review.openstack.org/#/c/141578/ - is that enough to fix this problem for now? i'd like to continue my happy-hacking weekend :) | 20:21 |
| dstufft | obviously you _can_ define a sort order for those, but it's just arbitrary with no semblance of actually having a meaning attached to that order | 20:21 |
| dstufft | and in fact, we have a place for opaque strings that don't have meaning attached to them, in the local version | 20:21 |
| clarkb | dstufft: to be fair the parse fail sort first has all of the same problems as ^ | 20:22 |
| clarkb | but that was apparently ok | 20:22 |
| clarkb | dstufft: right so I now know why I am so confused | 20:22 |
| *** dimsum__ has joined #openstack-infra | 20:22 | |
| clarkb | 440 was updated since I last read it completely removing the bits that we take advantage of | 20:22 |
| clarkb | so while we were 440 happy we are now not. | 20:22 |
| clarkb | which include source labels and rc versions | 20:23 |
| dstufft | clarkb: the difference is that the "parse fail sort first" isn't part of the PEP, we just added that in the packaging library, primarily for situations where someone _already_ has a version installed | 20:24 |
| *** armax has quit IRC | 20:24 | |
| dstufft | because we have to be able to sort the installed version and the candidate versions for techincal reasons | 20:24 |
| clarkb | and lifeless' version.py changes were largely written before the updates to pep440 ok. I have unraveled this in my head | 20:26 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Revert all of the semver patches https://review.openstack.org/141579 | 20:26 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Prefix git suffixes with + instead of . https://review.openstack.org/141580 | 20:26 |
| clarkb | mordred: the problem with ^ is it won't pass due to the sqlalchemy thing | 20:26 |
| mordred | sure | 20:26 |
| clarkb | mordred: however we can likely test it off to the side while we deal with that | 20:26 |
| mordred | but there's the code | 20:26 |
| mordred | yah | 20:27 |
| mordred | https://hg.python.org/peps/rev/59a0d31a1bc2 | 20:27 |
| * morganfainberg came to look at SQLAlchemy thing and see people are aware | 20:27 | |
| AJaeger | mordred: please don't start rewriting OpenStack in Go! ;( | 20:27 |
| clarkb | mordred: but ya now I know why I was so confused. we wrote all of this stuff just prior to nick changing it all | 20:27 |
| clarkb | AJaeger: I am ready to rewrite it in php | 20:27 |
| mordred | that change basically just pulled the rug out of all of our understand of what was going to be happening here | 20:27 |
| morganfainberg | mordred, hah @ the go comment. | 20:27 |
| mordred | also - really? removing rc for c? | 20:27 |
| *** social has quit IRC | 20:28 | |
| dstufft | we didn't remove rc | 20:28 |
| mordred | who is the world thinks of c as meaning rc? | 20:28 |
| mordred | yeah you did | 20:28 |
| clarkb | dstufft: yes that change above removed it | 20:28 |
| mordred | https://hg.python.org/peps/rev/59a0d31a1bc2 | 20:28 |
| dstufft | no we did not | 20:28 |
| mordred | ^^^^^ | 20:28 |
| dstufft | it moved it to normalization | 20:28 |
| dstufft | Pre-releases allow the additional spellings of alpha, beta, and rc for a, b, and c respectively. | 20:28 |
| fungi | fwiw, other packaging systems do sort alphanumeric strings in .-separated components after pure numeric and then proceed in c sort order (so dog sorts after cat because ord('c') < ord('d') in that case) | 20:28 |
| dstufft | I know what I wrote | 20:29 |
| clarkb | dstufft: then why remove that from the grammar? | 20:29 |
| mordred | fine whatever | 20:29 |
| mordred | I'm done arguing about this | 20:29 |
| dstufft | clarkb: the grammar reflects the "normalized" form | 20:29 |
| clarkb | Public version identifiers MUST comply with the following scheme: ... | 20:29 |
| mordred | I've indicated that I think it's crazy and nothing is going to change that. I will now continue working on unbreaking OpenStack | 20:30 |
| dstufft | In order to maintain better compatibility with existing versions there are a number of "alternative" syntaxes that MUST be taken into account when parsing versions. These syntaxes MUST be considered when parsing a version, however they should be "normalized" to the standard syntax defined above. | 20:30 |
| clarkb | dstufft: maybe that MUST should be removed? | 20:30 |
| * AJaeger remembers there was a similar change in rpm with sorting of version numbers that broke a few packages a couple of years ago ;( | 20:30 | |
| mordred | ruhe: yeah - I thnk that gets us past step one | 20:31 |
| clarkb | mordred: see comment on https://review.openstack.org/#/c/141580/ | 20:31 |
| dstufft | clarkb: it's fine to clarify things in an already accepted PEP yea, we can reword it if the wording is confusing | 20:31 |
| dstufft | 1.0rc1 is perfectly fine | 20:31 |
| mordred | clarkb: thanks | 20:32 |
| dstufft | it's not the normalized form, which tooling is supposed to prefer to emit normalized | 20:32 |
| clarkb | dstufft: that is what confused me. The statement basically says this is the only grammar accepted by public versions. Then later it says "we also accept these things" | 20:32 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Prefix git suffixes with + instead of . https://review.openstack.org/141580 | 20:32 |
| dstufft | clarkb: part of that is because the full grammar is not very meaningful looking | 20:32 |
| dstufft | https://github.com/pypa/packaging/blob/master/packaging/version.py#L162-L198 | 20:32 |
| mordred | yah - I follow what's going on there now | 20:33 |
| clarkb | dstufft: so maybe say this is the canonical normalized grammar and refer to exceptions below instead of you must conform to this grammar? | 20:33 |
| fungi | so the ".dev123" version specifiers we were using for unreleased version numbers at one point... were those semver-derived or earlier-pep440-revision? | 20:33 |
| dstufft | .dev123 is valid | 20:33 |
| clarkb | fungi: they are still pep 440 | 20:34 |
| fungi | k | 20:34 |
| *** amitgandhinz has joined #openstack-infra | 20:34 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: add support to git for changelog against branch https://review.openstack.org/134014 | 20:34 |
| dstufft | clarkb: I can make that update | 20:34 |
| *** armax has joined #openstack-infra | 20:34 | |
| *** armax has quit IRC | 20:34 | |
| fungi | though i suppose ideally pbr should start emitting only normalized pep440 version strings for better clarity | 20:35 |
| mordred | clarkb: so, I think we land the two patches from ruhe, then land and release pbr | 20:35 |
| mordred | fungi: right. | 20:35 |
| * fungi is mostly caught up with scrollback now | 20:35 | |
| mordred | fungi: https://review.openstack.org/141580 should fix pbr version emitting | 20:35 |
| dstufft | fungi: in an ideal PEP 440 centric viewpoint yes, and you can use the packaging library to do that if you want | 20:35 |
| clarkb | mordred: ya reviewing ruhe's changes now | 20:35 |
| *** shashankhegde has quit IRC | 20:35 | |
| dstufft | str(packaging.version.Version("some string")) | 20:35 |
| dstufft | wil give you a normalized form | 20:35 |
| fungi | dstufft: that's in stdlib now? | 20:35 |
| mordred | yeah, we can't use the packaging library unfortunately, because of setup_requires being evil | 20:35 |
| *** ryanpetrello has quit IRC | 20:36 | |
| dstufft | fungi: no, it's a third party lib that's designed to be easily bundled | 20:36 |
| fungi | ahh, right | 20:36 |
| clarkb | mordred: except it must exist in setuptools right? | 20:36 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add Mercurial plugin feature to jenkins-job-builder https://review.openstack.org/139136 | 20:36 |
| clarkb | mordred: which we will have access to | 20:36 |
| dstufft | it's bundled in setuptools yes, though it's "private" | 20:36 |
| mordred | dstufft: is it vendorerd in setuptools? | 20:36 |
| dstufft | but | 20:36 |
| fungi | pbr can only rely on stdlib because we don't want pbr to bootstrap something else to bootstrap pbr | 20:36 |
| fungi | though i guess we do rely on setuptools | 20:36 |
| dstufft | setuptools 8.0+ will return the version class from pkg_resources.parse_version | 20:36 |
| clarkb | fungi: no pbr depends on setuptools existing | 20:36 |
| fungi | i suspect we have to have a sane fallback behavior in the face of older setuptools which lacks that method though | 20:37 |
| clarkb | fungi: yup | 20:37 |
| *** pc_m has quit IRC | 20:37 | |
| dstufft | you can do parsed_version = pkg_resources.parse_version("some version"); if isinstance(parsed_version, tuple): fallback else: normalized = str(parsed_version) | 20:37 |
| mordred | what normalization do we need to do? the only thing we've got that's problematic at this point is the emission of .gXXXXXX | 20:38 |
| clarkb | ruhe: mordred should we update ruhe's patches with pip install -U pip<6 too? | 20:38 |
| clarkb | mordred: I think this is thoughts on rewriting lifeless' stack | 20:38 |
| mordred | oh - meh | 20:38 |
| clarkb | mordred: since a lot of that is superceded by this | 20:38 |
| dstufft | mordred: https://www.python.org/dev/peps/pep-0440/#normalization | 20:38 |
| clarkb | we can worry about it later | 20:38 |
| dstufft | that's not mandatory stuff though for emiting | 20:38 |
| dstufft | it's mandatory for parsing | 20:38 |
| clarkb | meh we can update pip install later if we need to | 20:39 |
| mordred | clarkb: ++ | 20:39 |
| mordred | do we have any devstack cores aroudn? | 20:39 |
| clarkb | now hopefully tox doesn't release | 20:39 |
| mordred | sdague: you online? | 20:39 |
| mordred | clarkb: oh dear god | 20:39 |
| *** armax has joined #openstack-infra | 20:39 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Allow multiple comment-added events in gerrit trig. https://review.openstack.org/124568 | 20:40 |
| clarkb | we also need to patch grenade | 20:40 |
| clarkb | and I don't think that we can do this without force merging something because lol | 20:40 |
| fungi | mordred: do you have the semver commit series handy and i'll repropose them? | 20:41 |
| * dtroyer just caught up with this… | 20:41 | |
| clarkb | I am looking at fixing grenade now | 20:41 |
| fungi | (or we can just let lifeless repropose them so gerrit still considers him the change owner) | 20:41 |
| clarkb | oh hrm grenade doesn't setuptools directly it must rely on the devstack change except that grenade doesn't use proposed devstack? | 20:42 |
| fungi | oh, i guess we can release on 141580 once it merges and then merge a revert of 141579 | 20:42 |
| dtroyer | grenade tries to let the two devstack's manage their own environments | 20:43 |
| clarkb | dtroyer: oh I see, so its likely that we need stable/juno backport for grenade to work | 20:43 |
| dtroyer | probably… | 20:44 |
| clarkb | so it is using proposed but only on the one branch which makes sense | 20:44 |
| *** amitgandhinz has quit IRC | 20:44 | |
| mordred | fungi: no, reset to 0.10.1, then cherry-picked all of the non-semver changes on top of it, then did a soft reset and committed that as the "revert" | 20:44 |
| mordred | fungi: so, I Think the easiest would be to just propose a revert of the revert patch | 20:44 |
| clarkb | let me propose backports of that change | 20:45 |
| *** amitgandhinz has joined #openstack-infra | 20:45 | |
| openstackgerrit | Merged openstack-infra/system-config: Pin version of setuptools https://review.openstack.org/141577 | 20:45 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add support for Gerrit Trigger Comment Contains Expression https://review.openstack.org/136605 | 20:45 |
| dstufft | clarkb: : /Users/dstufft/projects/setuptools/pkg_resources.py:2314: RuntimeWarning: 'pip (6.0.dev1.wat)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. It is recommend to migrate to PEP 440 compatible versions. | 20:48 |
| dstufft | clarkb: would that warning be acceptable? | 20:48 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: URL in Maven deploy is an optional field https://review.openstack.org/138586 | 20:48 |
| mordred | dstufft: yeah, that would probably be helpful | 20:50 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Reorganize tests/cmd https://review.openstack.org/134055 | 20:50 |
| AJaeger | dstufft: thanks! Would it make sense to add something like "It's sorted now as less than 0.0?" | 20:50 |
| clarkb | dstufft: +1 | 20:51 |
| dstufft | AJaeger: /Users/dstufft/projects/setuptools/pkg_resources.py:2315: RuntimeWarning: 'pip (6.0.dev1.wat)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions. | 20:51 |
| dstufft | better? | 20:51 |
| *** nikil22 has quit IRC | 20:51 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add support for Config File Provider to Maven Project module https://review.openstack.org/136225 | 20:51 |
| mordred | ++ | 20:52 |
| * AJaeger is happy, thanks dstufft | 20:52 | |
| clarkb | https://review.openstack.org/#/c/141581/ https://review.openstack.org/#/c/141582/ are devstack backports | 20:53 |
| clarkb | stable-maint: note that 141582 Is NOT a backport and nor should it be. If I get -2's onthat Iwill be very unhappy. | 20:53 |
| dstufft | https://github.com/jaraco/setuptools/pull/15 | 20:55 |
| ruhe | clarkb: re 141582. wouldn't unquoted <8.0 cause issues because of shell interpreting < as a redirect command? | 20:56 |
| clarkb | ruhe: gah you are correct :) | 20:56 |
| *** salv-orlando has joined #openstack-infra | 20:57 | |
| clarkb | note if we approve them in icehouse -> juno -> master order I don't think we need to force merge | 20:57 |
| clarkb | ruhe: thanks | 20:57 |
| *** AJaeger has quit IRC | 20:58 | |
| clarkb | who are devstack cores? dtroyer sdague mtreinish jeblair ianw ? | 20:58 |
| mordred | yah. dtroyer is online | 20:58 |
| *** hdd has joined #openstack-infra | 20:58 | |
| * mordred thinks this is one of those where waiting for 2 cores may not be necssary | 20:58 | |
| clarkb | +1 | 20:58 |
| clarkb | and ya if the icehouse change works then we approve that one, then recheck juno, approve juno, recheck master, approve master | 20:59 |
| dstufft | clarkb: https://bpaste.net/show/8243ae9b860f better? | 21:01 |
| *** salv-orlando has quit IRC | 21:01 | |
| clarkb | dstufft: ya I think that will help me parse at least | 21:02 |
| clarkb | dstufft: also quick question about the sort order thing | 21:02 |
| clarkb | dstufft: you negate the versions right so say I have 2.3.9.g1231231 installed. That becomes -1.2.3.9.g1231231 | 21:02 |
| clarkb | assuming that is installed | 21:02 |
| fungi | clarkb: fwiw the stable-maint team no longer has jurisdiction over devstack | 21:03 |
| *** armax has quit IRC | 21:03 | |
| ruhe | clarkb: is that change to stable/icehouse being tested by tempest from master? (just wondering) | 21:03 |
| fungi | clarkb: devstack cores should be able to approve as they see fit | 21:03 |
| clarkb | the next version is 2.3.10.g1231231 which becomes -1.2.3.10.g1231231 | 21:03 |
| clarkb | dstufft: would the next version be treated as lower not higher? | 21:03 |
| dtroyer | I didn't know they ever did? | 21:03 |
| dstufft | clarkb: sort of, it's not actually the version string that gets negated | 21:03 |
| dstufft | we compute a "sort key" | 21:03 |
| dstufft | which is a just a tuple | 21:04 |
| clarkb | fungi: oh thats good. I just had such a bad experience pushing fixes that were literally fixes to stable in the past and reviewers could not get over that it wasn't a backport | 21:04 |
| dstufft | and really it's just (-1, <previous sort key>) and (1, <pep 440 sort key>) | 21:04 |
| dstufft | and then we just rely on Python's tuple sorting | 21:04 |
| clarkb | dstufft: ok. But does that respect absolute values or ah ok | 21:04 |
| *** avozza is now known as zz_avozza | 21:04 | |
| dstufft | so 2.3.10.g1231231 will show as a newer version than 2.3.9.g1231231 | 21:04 |
| fungi | clarkb: there are now only 11 projects over which the legacy openstack-stable-maint team has control, and those will soon become project-specific stable-maint teams | 21:04 |
| fungi | (as of yesterday) | 21:05 |
| *** amotoki has quit IRC | 21:05 | |
| dstufft | clarkb: published the PEP update -> https://hg.python.org/peps/rev/047d121b118d | 21:05 |
| dstufft | it'll go live on the website at some point | 21:05 |
| dstufft | I think that cronjob is hourly | 21:05 |
| *** zz_avozza is now known as avozza | 21:05 | |
| clarkb | cool | 21:06 |
| clarkb | oh you know what icehouse may just work becuase of pinning | 21:08 |
| clarkb | so we may be able to merge both backports at the same time | 21:08 |
| dstufft | For whatever it's worth, we don't *like* breaking things, but we're in a situation where setuptools allows a lot of nonsense things like version numbers like "dog" and we're trying to standardize and exclude the "nonsense" things without breaking _too_ many things. We are testing our stuff against what's released on PyPI and we generally shoot for getting as close to 100% as possible.I didn't mean to sound as annoyed as I did at y'all, | 21:08 |
| dstufft | I have a headache and it's frustrating to get yelled at | 21:08 |
| *** xyang1 has joined #openstack-infra | 21:10 | |
| clarkb | ya I think its frustrating for all involved. apologies for our crankyness too | 21:13 |
| clarkb | the biggest miss here for me at least is we were 440 compliant and like 4 days after that code was written the pep changed without us knowing | 21:13 |
| clarkb | I think my next time feedback would be start with warnings in the code so that even people who think they are prepared get notified that they are not | 21:14 |
| fungi | likely we should have waited to implement pep 440 until it was approved | 21:14 |
| clarkb | fungi: except that we would've have been just as broken | 21:14 |
| fungi | and setuptools should have waited until long after pep 440 was approved to enforce it | 21:14 |
| fungi | but i get the eagerness to move on to new-shiny and not be stuck with a two-year deprecation cycle after the way forward is etched in stone | 21:15 |
| dstufft | I think setuptools is going to expand the "core contributors" to include me and I will probably start doing the setuptools releases at that point, which means they will likely start happening during the week | 21:15 |
| dstufft | I'm the only person who works on the packaging toolchain who gets work time to work on things, so that's why setuptools tends to get released on weekends | 21:16 |
| fungi | i just can't wait to see the inevitable explosion which will be the package-sig ml come monday | 21:16 |
| * fungi needs to make sure he is fully stocked on popcorn | 21:16 | |
| fungi | er, distutils-sig ml these days i mean | 21:18 |
| clarkb | mordred: also I propose we ditch the pbr semver stuff | 21:18 |
| clarkb | mordred: if pep440 is going to be enforced like this there is no sane way to have a not 1:1 set | 21:18 |
| clarkb | er not 1:1 set of versioning | 21:19 |
| mtreinish | clarkb: do you have links? I'll fast approve the patches | 21:19 |
| clarkb | mtreinish: https://review.openstack.org/#/c/141582/ https://review.openstack.org/#/c/141581/ and https://review.openstack.org/#/c/141578/ note that last one can't merge until the second one merges | 21:19 |
| dstufft | fungi: I don't think it's going to be that bad honestly. Like I said, we did check PEP 440 against PyPI and we were really close compat wise. Openstack's two problems were the SQLAlchemy specifier which is a fairly complex specifier and isn't the common case (even looking at the global requirements for openstack, out of ~200 some specifiers only one was broken) and the git hash in the version which I think pbr is one of the few reasons | 21:19 |
| dstufft | someone would have something like that | 21:19 |
| clarkb | I think the first two can merge now | 21:19 |
| fungi | dstufft: you checked that the version numbers registered on pypi were mostly pep-440-compliant, or that most packages on pypi use pep-440-compliant version specifiers for installing their dependencies? | 21:21 |
| fungi | those seem like very different scales of problems to identify | 21:21 |
| fungi | anyway, water under the bridge now. hopefully whatever pain is encountered is short-lived and catalyzes the community into more standardized versioning going forward, on and off pypi | 21:23 |
| clarkb | once the devstack changes merge next step is getting pbr merged and merging all updates to sqlalchemy that haven't merged then tagging things like oslo.do to use the valid sqlaclchemy version | 21:23 |
| *** zz_dimtruck is now known as dimtruck | 21:23 | |
| mtreinish | clarkb: ok, approved the stable branch versions | 21:23 |
| dstufft | fungi: the first one we checked everything by comparing all versions found on PyPI so we have exact numbers for that, the other one is harder and it was checked by going through github's code search looking for setup.py and requirements.txt files and I manually checked them | 21:23 |
| clarkb | mtreinish: thank you | 21:23 |
| dstufft | so the second one is a lot softer number | 21:23 |
| fungi | dstufft: got it. i figured that was far from easy (seems almost intractable even) | 21:24 |
| *** rkukura has quit IRC | 21:24 | |
| dstufft | fungi: we also checked to see that the packaging lib sorted the available versions on PyPI the same as setuptools did previously | 21:24 |
| dstufft | both with non PEP 440 versions included, and with them excluded | 21:24 |
| clarkb | mordred: you can tag a pbr release? | 21:24 |
| clarkb | I guess I might be able to too | 21:24 |
| dstufft | 96.45% of projects sorted the same when we included the non PEP 440 versions | 21:25 |
| dstufft | 99.88% sorted the same when we excluded them | 21:25 |
| fungi | dstufft: yep, seems carefully thought out. i still expect a number of corner cases to come up on the ml over the next week or two | 21:25 |
| mordred | clarkb: I can | 21:26 |
| fungi | in our case, we have enough dependencies that the transitive tree is just about guaranteed to fall within that 0.12% | 21:26 |
| dstufft | fungi: yea I'm sure there will be corner cases | 21:26 |
| clarkb | mordred: trying to figure out the things that need to happen as early as posible since I will be out on monday | 21:26 |
| mordred | clarkb: I'll get the pbr release cut today | 21:26 |
| clarkb | fungi: thankfully we can test for that by reverting the setuptools pins | 21:26 |
| *** avozza is now known as zz_avozza | 21:26 | |
| clarkb | fungi: so at least we will be aware | 21:26 |
| clarkb | (it is larissa's birthday so not negotiating out of that to fix stuff :) ) | 21:27 |
| *** zz_sabari is now known as sabari | 21:27 | |
| fungi | go celebrate | 21:27 |
| clarkb | fungi: I will on monday :) | 21:27 |
| mordred | nice part is - people have been wanting a pbr release for other features but we've been blocked on the lifeless patches | 21:27 |
| clarkb | sorry I meant monday is when I have to afk so I want to be helpful now | 21:27 |
| mordred | so this gets the pressure off of that work | 21:28 |
| dstufft | clarkb: mordred fungi FWIW we did not actually remove the source field from metadata 2.0, we just moved it from PEP 440 to PEP 426 because it's a metadata field not a version number part | 21:28 |
| dstufft | metadata 2.0 isn't done yet, that's going to be the enxt big effort I think after warehouse | 21:28 |
| mordred | nod | 21:28 |
| mordred | how's warehouse going? | 21:28 |
| *** armax has joined #openstack-infra | 21:28 | |
| dstufft | mordred: back burnered until pip 6 is done | 21:28 |
| dstufft | which i'm planning to release next week | 21:28 |
| clarkb | (which will also break us if we don't get this all fixed first. I think we can do it but depends on how bad transitive lib stuff is) | 21:29 |
| fungi | dstufft: i think the main reason for having the git sha was to help us solve a potential corner case where you had sdists built from parallel branches descending from a common latest tag and the number of commits on each branch was the same. i'm still not convinced that's actually common enough to warrant the added complexity though | 21:29 |
| dstufft | yea I just merged PEP 440 support into pip 6 | 21:29 |
| fungi | and there is an open bug against pbr for an option flag to allow you to turn off appending the git sha anyway | 21:30 |
| dstufft | fungi: for the record, the source field in metadata 2.0 is entirely informational, it's used as a "this release came from this version", pip and co won't use that field in it's version resolution algorithm at all | 21:30 |
| dstufft | came from this git hash* | 21:30 |
| dstufft | or whatever | 21:30 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Reorder imports to match hacking guidelines https://review.openstack.org/133465 | 21:30 |
| dstufft | clarkb: I'm flexible on what day it gets released, or can push it off an extra week too if y'all want more time | 21:31 |
| *** andreykurilin_ has joined #openstack-infra | 21:31 | |
| dstufft | I'd like to release it next week fi possible though | 21:31 |
| fungi | dstufft: right, which is why i think moving the git sha out of the version and into a metadata field doesn't really solve the problem i saw it being there to attempt to solve | 21:31 |
| clarkb | dstufft: would probably be good to see how far we get this weekend | 21:31 |
| clarkb | https://review.openstack.org/#/q/topic:openstack/requirements+status:open,n,z is basically the list of things that needs to merge | 21:31 |
| fungi | but i think it's an uncommon enough problem that i'm unconvinced it needs solving anyway | 21:31 |
| clarkb | then we need to cut releases for anything that is consumed via releases | 21:31 |
| clarkb | actually there are quite a few not sqlalchemy updates so that list isn't that bad | 21:32 |
| *** amitgandhinz has quit IRC | 21:32 | |
| clarkb | and devstack overriding requirements helps here | 21:32 |
| *** amitgandhinz has joined #openstack-infra | 21:32 | |
| *** rkukura has joined #openstack-infra | 21:32 | |
| clarkb | so we need it in anything consumed as a lib. python-*clients should not use sqlalchemy. I think that leaves the oslo projects | 21:32 |
| dstufft | fungi: yea, the + thing in versions isn't _techincally_ for that either, it's really designed for downstream people like ubuntu and co to mark when they patch an upstream thing or for people to fork an upstream project to mark they've forked it | 21:33 |
| dstufft | fungi: but it can be used in that context anyways since we just treat it as an opaque blob | 21:34 |
| ruhe | clarkb: ceilometer and glance are still using old sqlalchemy version constraints. probably will need to update them too? | 21:34 |
| clarkb | ruhe: ya, there are changes up for them. We don't need to focus on that though because global requirements overrides their requirements at install time | 21:34 |
| clarkb | ruhe: so we should do that but its a lower priorityc | 21:35 |
| ruhe | clarkb: i see. thanks for clarifying that | 21:35 |
| fungi | well, it may impact their unit test jobs et cetera | 21:35 |
| fungi | but not devstack-gate based jobs | 21:35 |
| clarkb | fungi: only once tox updates but yes | 21:36 |
| clarkb | a quick scan really shows oslo.db as the only one that I think we really need to worry about | 21:37 |
| *** amitgandhinz has quit IRC | 21:37 | |
| clarkb | its stable/juno requirements.txt need an update | 21:37 |
| dstufft | tox update shouldn't break anything until virtualenv is released | 21:38 |
| dstufft | that'll happen next week when I do pip | 21:38 |
| clarkb | http://git.openstack.org/cgit/openstack/requirements/tree/global-requirements.txt?h=stable/juno#n132 | 21:38 |
| clarkb | dstufft: do you want to propose your change ^ there too? | 21:38 |
| fungi | right, new tox vendoring new virtualenv vendoring new setuptools | 21:39 |
| dstufft | tox doesn't vendor virtualenv IIRC, it just depends on it | 21:39 |
| *** sabari is now known as zz_sabari | 21:39 | |
| clarkb | probably to >=0.8.4 ? | 21:39 |
| *** otter768 has joined #openstack-infra | 21:39 | |
| dstufft | clarkb: I'm not sure how to do that on another branch | 21:39 |
| dstufft | I only ever do just ``git review`` | 21:39 |
| clarkb | dstufft: its easy can walk you though it if you want | 21:39 |
| dstufft | ok | 21:39 |
| *** shakamunyi has quit IRC | 21:40 | |
| clarkb | dstufft: basically `git remote update ; git checkout -b stable/juno origin/stable/juno ; git checkout -b fix-sqlalchemy ; make edits ; commit ; git review stable/juno` | 21:40 |
| dstufft | clarkb: do you want the more expressive form of the specifier that includes a bunch of != so that you get the same versions that allows? | 21:40 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Support copyartifact build selector param https://review.openstack.org/133774 | 21:40 |
| clarkb | dstufft: if that is possible I think we should do that for juno since that is a stable branch | 21:41 |
| clarkb | dstufft: the change you made for kilo/master is fine because its the dev version but for the stable branch we should make it as 1:1 as possible | 21:41 |
| dstufft | get's the same versions assuming sqlalchemy doens't release a 0.9.1.1 or something | 21:41 |
| dstufft | * | 21:41 |
| clarkb | dstufft: ya I think we can live with that corner case | 21:41 |
| *** otter768 has quit IRC | 21:43 | |
| *** barra204 has quit IRC | 21:45 | |
| clarkb | dhellmann: for when you see IRC can we have a new release of oslo.db? we need it for updated sqlalchemy requirements. We will need the same for the juno branch too once we get the update on the juno branch | 21:45 |
| *** Masahiro has joined #openstack-infra | 21:45 | |
| dstufft | https://review.openstack.org/#/c/141584/ | 21:46 |
| *** jamespage_ has joined #openstack-infra | 21:46 | |
| clarkb | that will likely fail testing until the juno devstack change gets in | 21:47 |
| clarkb | (again because of libs consumed from releases) | 21:47 |
| clarkb | but I can recheck it once the devstack change merges and I am +2 | 21:47 |
| dstufft | ok | 21:49 |
| dstufft | i double checked it would do as it looks on both old pip and pip 6 with both old setuptools and new | 21:49 |
| dtroyer | devstack stable/juno isn't happy with check-grenade-dsvm-ironic-sideways… | 21:49 |
| *** Masahiro has quit IRC | 21:50 | |
| dtroyer | last successful run appears to be Dec 5 | 21:50 |
| clarkb | dstufft: perfect thank you | 21:50 |
| clarkb | dtroyer: hrm | 21:50 |
| dstufft | clarkb: for whatever it's worth, at some point in the future I want to see about adding grouping and OR operations to specifiers | 21:50 |
| clarkb | adam_g: devananda do you happen to be around? | 21:50 |
| mtreinish | dtroyer: oh, I know that, there's a tempest bug | 21:50 |
| mtreinish | I think the fix was in progress, let me fast approve it | 21:51 |
| dtroyer | the IPv6 one? | 21:51 |
| clarkb | mtreinish: it cannot merge :) | 21:51 |
| mtreinish | yep | 21:51 |
| dstufft | because what you were trying to do with that specifier was something like (>=0.8.4 AND <=0.8.99) OR (>=0.9.7 AND <=0.9.99) | 21:51 |
| clarkb | so with juno we may have wedged ourselves | 21:51 |
| clarkb | dstufft: ya | 21:51 |
| mtreinish | clarkb: d'oh, yeah that's what it sounds like | 21:51 |
| mtreinish | dtroyer: https://review.openstack.org/#/c/141456/ | 21:52 |
| clarkb | I think we can force merge the juno devstack change if the devstack cores are ok with it | 21:52 |
| dstufft | clarkb: setuptools 8 (and pip 6) includes some new specifiers too btw, like ~= and the ability to do ==1.* and such | 21:52 |
| clarkb | then we can worry about fixing tempest for ironic sideways | 21:52 |
| mtreinish | clarkb: go ahead, I'm fine with that | 21:52 |
| dtroyer | clarkb: I'm good with that | 21:52 |
| dstufft | (can't use them until you're happy making those min versions tho) | 21:52 |
| clarkb | dstufft: ~= is basically match any "smaller" identifiers? | 21:53 |
| dstufft | newer, it's the semver matcher basically | 21:53 |
| clarkb | rgr | 21:53 |
| dstufft | ~=2.3 is >=2.3 AND < 3 | 21:53 |
| dstufft | more or less | 21:53 |
| clarkb | fungi: mordred: any opposition to me merging https://review.openstack.org/#/c/141581/ ? | 21:53 |
| clarkb | I did propose it maybe one of you want to do that instead :) | 21:54 |
| clarkb | though ruhe wrote it | 21:54 |
| *** tnovacik has quit IRC | 21:56 | |
| mordred | clarkb: I can do it | 21:56 |
| mordred | clarkb: oh - that's a force merge, right? | 21:56 |
| mordred | clarkb: I thought we thought that we didnt' need a force merge? | 21:57 |
| clarkb | mordred: yes, it needs a force merge because ironic sideways is failing which requires a tempest fix which can't merge until setuptools is pinned on juno | 21:57 |
| clarkb | mordred: we don't need it other than ironic sideways is failing | 21:57 |
| clarkb | mordred: also you can see that that change makes the other jobs pass | 21:57 |
| clarkb | so it works just not in that psecific case where tempest fails on ironic sideways | 21:58 |
| mordred | cool | 21:58 |
| mordred | I'll nija it | 21:58 |
| mordred | clarkb: done | 21:59 |
| clarkb | mordred: thank you I am rechecking the master change now | 21:59 |
| clarkb | mtreinish: now you can work on getting ironic fixed which we likely need as there will be many changes in the juno branch to update sqlalchemy requirement | 22:00 |
| mtreinish | clarkb: I already fast approved the ironic fix :) | 22:01 |
| clarkb | woot | 22:01 |
| mtreinish | it was just a skip, because ironic doesn't support specifying network | 22:01 |
| clarkb | that is easy | 22:01 |
| clarkb | I have rechecked dstufft's requirements juno change now too | 22:06 |
| dstufft | cool | 22:07 |
| dstufft | -> goes to get dinner | 22:07 |
| mordred | dstufft: heh. I'm still workingon morning coffee | 22:07 |
| clarkb | mordred: once devstack master change gets in (mtreinish can haz approval on that so it goes straight ot gate if tests pass?) its on to fixing pbr | 22:08 |
| mtreinish | clarkb: +A | 22:09 |
| mordred | clarkb: fixing pbr should just be rechecking the semver revert, then landing, then releasing, yeah? | 22:09 |
| clarkb | mordred: ya | 22:09 |
| clarkb | mordred: except then we need to go make everything use that version of pbr | 22:09 |
| mordred | nah - the version spec is wide open for pbr | 22:10 |
| clarkb | mordred: awesome | 22:10 |
| mordred | it's just <1.0 | 22:10 |
| mordred | so we're fine | 22:10 |
| clarkb | cool so other than updating requirements in all the things I think we are in really good shape | 22:11 |
| mordred | yah | 22:11 |
| mordred | this is one of the easier break-the-world's we've had :) | 22:11 |
| mordred | either that or we're just getting better at it | 22:11 |
| clarkb | dstufft: tl;dr I think there is a good chance we won't need you to delay pip release if you give us a bit of time | 22:11 |
| clarkb | mtreinish: oh crud tempest is branchless :) | 22:13 |
| clarkb | mtreinish: so we can't fix tempest until after devstack is updated | 22:13 |
| clarkb | oh well that just means a recheck later today | 22:14 |
| clarkb | in ~1.5 hours | 22:14 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add --average option to sql2subunit cli https://review.openstack.org/132119 | 22:14 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Correct writing of timestampes in write_test() for sql2subunit https://review.openstack.org/141585 | 22:14 |
| mtreinish | clarkb: oh, yeah forgot about that | 22:14 |
| mtreinish | I'll try to remember to do that later tonight | 22:14 |
| clarkb | me too o/ | 22:14 |
| clarkb | so we were super duper extra wedged on that | 22:15 |
| clarkb | it was a triangle where devstack juno needed tempest master, tempest master needed devstack master, devstack master needed devstack juno | 22:16 |
| clarkb | mordred: ^ that is the mess that you untangled :) | 22:16 |
| * mordred is helpful | 22:16 | |
| mtreinish | ooh, yeah that's a doozy | 22:16 |
| *** zz_avozza is now known as avozza | 22:17 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Correct writing of timestampes in write_test() for sql2subunit https://review.openstack.org/141585 | 22:18 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add --average option to sql2subunit cli https://review.openstack.org/132119 | 22:18 |
| *** xyang1 has quit IRC | 22:20 | |
| clarkb | mordred: how does this affect installs that are just shortsha1 versioned? | 22:23 |
| clarkb | I think it should break int he same way as before so likely not something to worry about | 22:23 |
| *** avozza is now known as zz_avozza | 22:23 | |
| clarkb | mordred: `git describe --always` on a repo with no tags reports short sha which is not valid under 440 either | 22:26 |
| *** liusheng has quit IRC | 22:26 | |
| clarkb | mordred: but maybe that is desireable | 22:27 |
| *** liusheng has joined #openstack-infra | 22:27 | |
| clarkb | hrm is 0+g1231231 < 0.0.1 ? | 22:27 |
| clarkb | if so maybe we do something like that | 22:27 |
| mtreinish | clarkb: I think it is | 22:29 |
| mordred | yeah - we've never really expected that short-sha would be compat - so actually the pep440 change I think does the right thing | 22:29 |
| * mtreinish tries to remember the last time he fought the auto versioning in pbr | 22:29 | |
| clarkb | well in the case of no tag versions pbr gives you 1231231 | 22:29 |
| clarkb | which is going to be confusing making (and honestyl was in the past too) | 22:29 |
| mordred | k.I can make that change too | 22:30 |
| clarkb | but if we make that version 0+g1231231 I wonder if that works | 22:30 |
| mordred | 0+g123123 is fine | 22:30 |
| mordred | we can test with packaging library | 22:30 |
| clarkb | mordred: ok as long as it will work | 22:30 |
| clarkb | hrm actually | 22:30 |
| clarkb | 0.0.0.numcommitssincefirstcommit+g1231231 might be more pbrish | 22:31 |
| clarkb | in any case I Think we can worry about this particular scenario later | 22:31 |
| Mithrandir | 0~$(date +%s)+g123456 | 22:32 |
| clarkb | or even 0+numcommits.g1231231 | 22:32 |
| Mithrandir | 0.0~$(date +%s) | 22:32 |
| *** amitgandhinz has joined #openstack-infra | 22:33 | |
| mordred | >>> v=version.Version('0+g123123') | 22:33 |
| mordred | >>> z=version.Version('0.1.1') | 22:33 |
| mordred | >>> v<z | 22:33 |
| mordred | True | 22:33 |
| Mithrandir | ~ sorts as less than the empty string in deb and rpm world, + doesn't. | 22:33 |
| clarkb | Mithrandir: I don't think ~ is accepted by pep 440 | 22:33 |
| clarkb | since it is an operator | 22:33 |
| mordred | right. ~ is out | 22:33 |
| Mithrandir | operator? How is ~ an operator in version strings? | 22:34 |
| mordred | version strings will have to be translated from python to deb/rpm | 22:34 |
| clarkb | Mithrandir: its an operator in requirements | 22:34 |
| mordred | there is literally no string that works for all the systems | 22:34 |
| Mithrandir | :-/ | 22:34 |
| Mithrandir | mordred: sadness. | 22:34 |
| Mithrandir | but such is life. | 22:34 |
| mordred | Mithrandir: yah | 22:34 |
| * mordred stabs self in face | 22:34 | |
| clarkb | mordred: I think we want num of commits in there so that if you add a second commit it will always be a higher version | 22:35 |
| clarkb | mordred: so terrible idea is 0+numcommitsg1231231 | 22:35 |
| clarkb | eg 0+10g1231231 | 22:35 |
| mordred | yah | 22:35 |
| *** hdd has quit IRC | 22:35 | |
| Mithrandir | I'd recommend putting a timestamp in there, tbh. | 22:36 |
| clarkb | except that won't work | 22:36 |
| clarkb | because alpha sort | 22:36 |
| *** fandi has quit IRC | 22:36 | |
| clarkb | Mithrandir: except timestamps don't really mean much in git | 22:36 |
| mordred | I think we need to do: if "." not in $(git describe --always): prepend 0.0.0 | 22:36 |
| Mithrandir | clarkb: oh, timestamp and the commit hash, sure. | 22:36 |
| Mithrandir | clarkb: but timestamps mean something to humans. | 22:36 |
| clarkb | Mithrandir: but they don't mean things in git sorting | 22:36 |
| clarkb | which is really what we are trying to express onto pep 440 | 22:37 |
| *** stevemar has joined #openstack-infra | 22:37 | |
| Mithrandir | maybe I should read the pep and then tear my hair out so I understand where you're coming from at least. | 22:37 |
| *** amitgandhinz has quit IRC | 22:37 | |
| clarkb | so that if I have commits A <- B <- C that we also get version sorting A<B<C | 22:38 |
| clarkb | and I don't think timestamps give that to us in a git world (because git preserves timestamps when you move stuff around | 22:38 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Make a PEP440-ish version for non-tagged repos https://review.openstack.org/141586 | 22:39 |
| mordred | clarkb: ^^ what about that? | 22:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add new db api methods for getting test data from runs https://review.openstack.org/141547 | 22:39 |
| Mithrandir | clarkb: not the commit timestamp, I think? Maybe the authorship timestamp. | 22:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Refactor sql2subunit to use get_tests_run_dicts_from_run_id https://review.openstack.org/141548 | 22:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Correct writing of timestampes in write_test() for sql2subunit https://review.openstack.org/141585 | 22:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add --average option to sql2subunit cli https://review.openstack.org/132119 | 22:40 |
| clarkb | Mithrandir: I don't think either will work | 22:40 |
| clarkb | Mithrandir: especially with how we gerrit where stuff gets in at weird times | 22:40 |
| clarkb | mordred: I think that is a reasonable way to do it | 22:40 |
| clarkb | mordred: it won't work 100% of the time but it should work in a pbr world | 22:40 |
| dstufft | clarkb: 0+gwhatever is < 0.0.1 | 22:41 |
| *** stevemar has quit IRC | 22:41 | |
| clarkb | mordred: or maybe we devN? | 22:41 |
| Mithrandir | + isn't allowed according to pep 440 | 22:41 |
| clarkb | Mithrandir: it is | 22:41 |
| clarkb | Mithrandir: (don't worry I have lready been confused by this too :) ) | 22:42 |
| Mithrandir | oh, I'm reading the local version thing. | 22:42 |
| Mithrandir | sorry. | 22:42 |
| clarkb | mordred: oh wait + does the right thing if you do it like this: 0+10.123abcd | 22:43 |
| clarkb | mordred: the 10 will be treated as an integer and they split on .'s | 22:43 |
| clarkb | mordred: so I think what we actually want is something like ^ | 22:43 |
| clarkb | 0+10.g123abcd | 22:44 |
| clarkb | will always be less than 0+11.g012abc | 22:44 |
| dstufft | or 0.dev10+g1234 | 22:44 |
| clarkb | ya or we could use dev | 22:44 |
| * dstufft afks again | 22:44 | |
| clarkb | if we go to devN we should update all cases to use devN | 22:46 |
| clarkb | which is maybe best decided by what is old setuptools vs new setuptools look like with the different setups | 22:47 |
| Mithrandir | hmm, ~ isn't an operator, ~= is, but yeah. | 22:49 |
| * Mithrandir tears out hair. | 22:49 | |
| *** Longgeek has joined #openstack-infra | 22:52 | |
| *** shashankhegde has joined #openstack-infra | 22:53 | |
| *** jamielennox|away is now known as jamielennox | 22:55 | |
| *** esker has joined #openstack-infra | 22:55 | |
| clarkb | ok I am being called away to food. Will check back in later to see if tempest/pbr need rechecking | 22:56 |
| *** Longgeek has quit IRC | 22:56 | |
| clarkb | any thoughts on approving https://review.openstack.org/#/c/141584/1 too? I can do that when I get back if no one beats me to it | 22:57 |
| *** dimtruck is now known as zz_dimtruck | 22:58 | |
| ZZelle_ | Hi everyone | 23:01 |
| ZZelle_ | It seems there is a trouble during cirros image download by CI hosts | 23:01 |
| dstufft | Mithrandir: We didn't use ~ because of compatability with Debian, because + sorts _after_ | 23:03 |
| fungi | ZZelle_: i hope not--we pre-cache the guest images devstack uses to keep it from downloading them from the internet at all | 23:03 |
| dstufft | like 1.0+anything > 1.0 | 23:03 |
| dstufft | we didn't use - because it's what setuptools used for post releases | 23:03 |
| ZZelle_ | fungi, http://logs.openstack.org/57/141057/4/check//check-tempest-dsvm-neutron-pg/8bd823d/logs/screen-g-api.txt.gz | 23:03 |
| dstufft | we were running out of characters | 23:03 |
| Mithrandir | dstufft: yes, so ~ should be have been required for prereleases. | 23:03 |
| dstufft | we don't use ~ to demarcate pre-releases | 23:03 |
| dstufft | we have names for them | 23:03 |
| dstufft | 1.0a1 | 23:04 |
| dstufft | 1.0b1 | 23:04 |
| dstufft | 1.0c1 | 23:04 |
| ZZelle_ | fungi, there is the same trouble with cinder/neutron tests using devstack | 23:04 |
| dstufft | 1.0.dev0 | 23:04 |
| fungi | Mithrandir: i believe in pep440 it's 1.2.3a4 to express something along the lines of debian 1.2.3~4 or 1.2.3~a4 | 23:04 |
| dstufft | etc | 23:04 |
| Mithrandir | I think having "a", "b", "c" and the corresponding alpha/beta/rc magically in the middle of a string changing the sort order is insane. | 23:04 |
| *** mwagner_lap has joined #openstack-infra | 23:04 | |
| fungi | since 1.2.3a4 < 1.2.3 | 23:04 |
| ZZelle_ | fungi, "2014-12-13 22:04:43.453 505 ERROR glance.registry.client.v1.client [8dbd043f-37d4-450e-acd3-dddab94c732c f369f106b34d4a1a9d9bdadb40bfc130 34fb041fb9c84a12ac635083ee9bce72 - - -] Registry client request GET /images/cirros-0.3.2-x86_64-uec-ramdisk raised NotFound" | 23:04 |
| Mithrandir | fungi: except much less expressive. | 23:04 |
| dstufft | Mithrandir: *shrug*, basically zero people use anything but that in Python land | 23:04 |
| dstufft | or anywhere else I've seen tbh | 23:04 |
| dstufft | debian is the only place that uses ~ that i've seen | 23:05 |
| fungi | Mithrandir: i think that ship sailed 10+ years ago. pep440 is just trying to codify current usage and make as much sanity around it as can be found | 23:05 |
| * dstufft goes to eat | 23:05 | |
| Mithrandir | dstufft: it's supported in not-ancient RPM too. | 23:05 |
| dstufft | Mithrandir: cool, two linux distros against 15 years of legacy behavior | 23:05 |
| Mithrandir | and well, 0~$(date +%s)+githash is pretty common for "this is a really new thing" | 23:05 |
| fungi | yeah, most of the distro packaging systems are copying the dpkg ~ usage now, but that's a pretty recent event | 23:05 |
| Mithrandir | where pretty recent is ten years old. :-P | 23:06 |
| fungi | yep | 23:06 |
| Mithrandir | (give or take, I didn't check when ~ was added to policy) | 23:06 |
| fungi | Mithrandir: well, 10 years for debian. i mean only recently have non-debian-derived distros been following suit | 23:06 |
| *** ChanServ has quit IRC | 23:07 | |
| clarkb | ZZelle_ thats the setuptools issue | 23:08 |
| ZZelle_ | clarkb, ok | 23:08 |
| clarkb | it breaks swift which breaks glance | 23:08 |
| *** jamielennox is now known as jamielennox|away | 23:08 | |
| *** shashankhegde has quit IRC | 23:08 | |
| *** ChanServ has joined #openstack-infra | 23:10 | |
| *** sendak.freenode.net sets mode: +o ChanServ | 23:10 | |
| Mithrandir | as an aside, it also seems like pep440 is buggy in that it says that a component of 0 is the lowest value in the version ordering, then proceeds to talk about a0 being lower. | 23:10 |
| Mithrandir | or a1 or whatever. | 23:10 |
| *** zz_dimtruck is now known as dimtruck | 23:11 | |
| fungi | Mithrandir: non-negative numbers less than 0. i just thought of a paper i want to submit to the journal of number theory | 23:12 |
| Mithrandir | fungi: sure, 0~1 is a non-negative version that's less than 0. | 23:12 |
| Mithrandir | (this broke imports in launchpad at some point there was something we tried to sync from debian with a version of 0~$(date)+sha1 or whatever. | 23:13 |
| clarkb | fungi does that go in with the 0 is (not) an Integer papers? | 23:13 |
| Mithrandir | and it went "huh", since it compared with 0. | 23:13 |
| fungi | clarkb: probably, yeah. now i want to go back to roman numerals | 23:14 |
| clarkb | er sorry | 23:14 |
| clarkb | natural number | 23:14 |
| Mithrandir | I~V ? :-P | 23:14 |
| Mithrandir | I'm not sure how that's improving things. :-P | 23:14 |
| dstufft | can't make it worse | 23:14 |
| Mithrandir | you can't express 0, I guess that might make it.. otherwise confusing. | 23:14 |
| dstufft | setuptools 9 will enforce the new roman numerals based PEP | 23:14 |
| fungi | tordinal numbers | 23:14 |
| * fungi makes up terms | 23:15 | |
| dstufft | Mithrandir: that'll just make it easier to prevent people from keeping 0.X forever | 23:15 |
| Mithrandir | true dat. | 23:15 |
| Mithrandir | did the romans have floating point, even? | 23:15 |
| clarkb | fractions | 23:15 |
| fungi | it was all fractions iirc | 23:16 |
| dstufft | the romans were all about continous deplyoment | 23:16 |
| clarkb | III/V | 23:16 |
| Mithrandir | ok, then it's _really_ not improving things | 23:16 |
| *** stevemar has joined #openstack-infra | 23:16 | |
| fungi | iii/iv sorts after v/ix | 23:16 |
| *** dimtruck is now known as zz_dimtruck | 23:17 | |
| *** ChanServ has quit IRC | 23:24 | |
| *** hdd has joined #openstack-infra | 23:24 | |
| *** zz_avozza is now known as avozza | 23:25 | |
| *** hdd has quit IRC | 23:25 | |
| *** esker has quit IRC | 23:26 | |
| clarkb | devstack master merged. time to recheck tempest and pbr | 23:27 |
| *** ChanServ has joined #openstack-infra | 23:28 | |
| *** sendak.freenode.net sets mode: +o ChanServ | 23:28 | |
| clarkb | welcome back chanserv | 23:31 |
| *** ChanServ has quit IRC | 23:31 | |
| fungi | clearly they're still futzing around with their atheme server | 23:32 |
| clarkb | mordred pep8 fails on pbr changes | 23:32 |
| clarkb | the reverts apparently undid some pep8 stuff | 23:33 |
| lifeless | mordred: reverting semver? | 23:34 |
| clarkb | lifeless setuptools is enforcing pep440 now | 23:34 |
| lifeless | mordred: I think a single patch is needed to fix the version generation | 23:34 |
| *** Masahiro has joined #openstack-infra | 23:34 | |
| lifeless | mordred: reverting is really a poor idea IMO because it will allow the stable branch headache we had before to come back. | 23:34 |
| lifeless | mordred: so please don't do it that way | 23:35 |
| clarkb | lifeless except we also need to relwase and it wasnt ready for other reasons iirc | 23:35 |
| lifeless | clarkb: no, it was just one patch pending | 23:35 |
| clarkb | so maybe release based on 0.10.0 and keep semver on master | 23:35 |
| clarkb | as an alternative | 23:35 |
| lifeless | clarkb: which was the one to change dev versions to be in the same timeline as pre-release versions | 23:36 |
| *** ChanServ has joined #openstack-infra | 23:36 | |
| *** sendak.freenode.net sets mode: +o ChanServ | 23:36 | |
| lifeless | e.g. 1.2.3.b1.0dev1 rather than 1.2.3.0dev5 | 23:36 |
| lifeless | clarkb: thats _all_ | 23:36 |
| clarkb | we also need to fix the source index things | 23:37 |
| lifeless | source index? | 23:37 |
| clarkb | .g1231231 is invalid | 23:38 |
| clarkb | so we need to make it +g1231231 and parse that properly | 23:38 |
| clarkb | also drop rc in your normalization as 'c' is normalized now | 23:38 |
| *** Masahiro has quit IRC | 23:38 | |
| lifeless | ok, so do that, rather than reverting everything else | 23:39 |
| lifeless | my concern is that I'll have more work to do than just doing these two minor changes now | 23:40 |
| clarkb | or its like a 3 line diff atop 0.10.0 to not be completely broken | 23:40 |
| lifeless | but I have to leave in 90m to drive to dunedin, so I can't do it now | 23:40 |
| *** otter768 has joined #openstack-infra | 23:40 | |
| clarkb | lifeless I think your changes can ve mostly dropped now | 23:40 |
| clarkb | because setuptools | 23:40 |
| lifeless | one 3 lines patch vs 2 3 line patches? | 23:40 |
| lifeless | what has setuptools done that helps us ? | 23:40 |
| clarkb | they normalize and enforce versions now | 23:41 |
| clarkb | which is what breaks us | 23:41 |
| clarkb | but is also what pbr wanted to do | 23:41 |
| lifeless | which is about 5% of the work thats being reverted | 23:41 |
| clarkb | it is? | 23:41 |
| lifeless | yeah | 23:41 |
| clarkb | I think semver is basically check setuptools increment done now | 23:42 |
| lifeless | I think you're wrong | 23:42 |
| clarkb | since apparently they grok semver to anextenttoo | 23:42 |
| clarkb | via ~= | 23:42 |
| lifeless | the version normalisation bit, sure. Figure out what to increment from git - nope; offer the nice API for __version__ etc inside modules - no | 23:42 |
| *** shakamunyi has joined #openstack-infra | 23:43 | |
| *** barra204 has joined #openstack-infra | 23:43 | |
| *** EmilienM|pto is now known as EmilienM | 23:43 | |
| clarkb | anyways I looked at version.py and noped out because weekend | 23:44 |
| clarkb | its a lot more complicated conceptually than 0.10.0 | 23:44 |
| lifeless | fine, whatever. I don't have time to deal with this; if my work is reverted I'll do whatever is lowest cost engineering to get back to the current state - be that pester mordred to deal with the fallout his patch chain creates, or fork pbr or whatever, because annual leave. | 23:44 |
| *** otter768 has quit IRC | 23:44 | |
| dstufft | clarkb: for the record you won't be able to use ~= until you're happy mandating a min version on setuptools 8 and pip 6 | 23:44 |
| lifeless | clarkb: personally, setuptools broke things? setuptools should rever. | 23:45 |
| lifeless | clarkb: the migration plan for PEP-440 AIUI was always to be *graceful* | 23:45 |
| dstufft | it is graceful for most situations | 23:45 |
| lifeless | clarkb: and this - us scrambling - is so not graceful. | 23:45 |
| clarkb | right setuptools forced the issue | 23:45 |
| clarkb | you cannot be graceful anymore | 23:45 |
| lifeless | dstufft: in what way is causing us to throw out 6 months of work graceful? | 23:45 |
| lifeless | alternative fix: blacklist setuptools >= 7.0.0 | 23:46 |
| dstufft | lifeless: I have no idea what the 6 months of work is, the problems were that you had a somewhat insane version specifier and that you needed to switch .githash to +githash | 23:46 |
| clarkb | lifeless thats the immediate fix yes | 23:46 |
| *** salv-orlando has joined #openstack-infra | 23:46 | |
| clarkb | but we have to do that to pip and virtualenv if we dont fix asap | 23:47 |
| clarkb | and blacklisting all the things is :( | 23:47 |
| clarkb | oh and tox | 23:47 |
| *** andreykurilin_ has quit IRC | 23:48 | |
| dstufft | the things that were broken wern't an accident. y'all were outliers and relying on things that we explicitly wanted to disallow | 23:48 |
| clarkb | well | 23:48 |
| clarkb | they were allowed back in june | 23:48 |
| lifeless | dstufft: which you knew we a) were relying on, b) had not yet fixed. | 23:49 |
| clarkb | when we started our process of being compliant | 23:49 |
| dstufft | clarkb: I don't think there was ever a version of PEP 440 that allowed 1.0.githash | 23:49 |
| clarkb | dstufft there was | 23:49 |
| clarkb | same diff that removed rc from grammar | 23:49 |
| lifeless | dstufft: 1.0.g$HASH FTR. And yes, there was. | 23:49 |
| lifeless | dstufft: we cross referenced everything against the PEP when we wrote our spec. | 23:50 |
| clarkb | yup | 23:50 |
| clarkb | but then pep drastically changed as it relates to us | 23:50 |
| dstufft | clarkb: that diff was 1.0-githash | 23:50 |
| dstufft | not 1.0.githash | 23:50 |
| clarkb | no 1.0.ggiyhash was allowed | 23:50 |
| dstufft | acutally it wasn't even 1.0-githash, it was 1.0-integer | 23:50 |
| clarkb | had to start with not digit iirc | 23:51 |
| lifeless | dstufft: and yes it was a moving target, we just didn't expect to be penalised by setuptools deciding anything previously valid and not anymore was broken in a release with NO DEPRECATION PERIOD | 23:51 |
| dstufft | clarkb: https://hg.python.org/peps/rev/59a0d31a1bc2 can show me where? because I don't see it | 23:51 |
| *** SumitNaiksatam has quit IRC | 23:52 | |
| lifeless | dstufft: IMO forget the pep for a second. setuptools worked. Now it doesn't. It is deliberate. Where was the deprecation period ?! | 23:53 |
| clarkb | dstufft source labels | 23:53 |
| *** SumitNaiksatam has joined #openstack-infra | 23:53 | |
| dstufft | clarkb: source labels were never part of the version tag | 23:54 |
| fungi | lifeless: so anyway i think the revert can be dealt with using a temporary backport branch instead. basically we need to release a pbr which switches .g to +g without necessarily also releasing the current semver work in progress along with it. i think if we branch from the last tag, apply the trivial patch, tag that and then delete the backport branch and merge that tag back into master we'll be | 23:54 |
| fungi | fine? | 23:54 |
| dstufft | lifeless: there wasn't one, because the technical feasibility of doing one is dificult, and this only breaks a handful of projects that we were able to determine | 23:54 |
| clarkb | fungi ya I think that is a reasonable step | 23:55 |
| fungi | that also avoids muddying the git history with a largeish rollback followed by a revert of the same | 23:56 |
| clarkb | fungi it was my original suggestion to mordred | 23:56 |
| fungi | ahh, i probably skimmed past that | 23:57 |
| dstufft | lifeless: in particular, one side effect of this change is that for many people, setuptools is now acting in accodrance with how they _expected_ it to act, so for many people this is fixing bugs not breaking things, and there are more situations like that according to our surveying than situations where this broke things, so adding a warning is going to confuse those people | 23:57 |
| lifeless | said warning would have said 'something odd is happening here and its going to be prohibited in future' - I don't see why that would confuse them | 23:58 |
| *** pcrews has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!