| opendevreview | Takashi Kajinami proposed openstack/placement master: Skip functional tests on pre-commit-config.yaml update https://review.opendev.org/c/openstack/placement/+/933191 | 01:24 |
|---|---|---|
| opendevreview | Takashi Kajinami proposed openstack/placement master: Skip functional tests on pre-commit config update https://review.opendev.org/c/openstack/placement/+/933191 | 01:24 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Skip functional tests on pre-commit config update https://review.opendev.org/c/openstack/nova/+/933192 | 01:28 |
| opendevreview | Takashi Kajinami proposed openstack/placement master: Skip functional tests on pre-commit config update https://review.opendev.org/c/openstack/placement/+/933191 | 01:41 |
| opendevreview | Takashi Kajinami proposed openstack/placement master: Skip functional tests on pre-commit config update https://review.opendev.org/c/openstack/placement/+/933191 | 01:41 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Skip functional tests on pre-commit config update https://review.opendev.org/c/openstack/nova/+/933192 | 01:46 |
| sean-k-mooney | atmark: we dont actully supprot changign the image_type on an exsiting comptue node and there is no officall supprot form moving a vm form one storage backend to annother | 02:51 |
| sean-k-mooney | it can be doen in some cases via shelve and unshleve. | 02:51 |
| songwenping | sean-k-mooney, hi, the bug 2059809 doesnot effect V and previsous versions? | 03:08 |
| sean-k-mooney | https://bugs.launchpad.net/nova/+bug/2059809 | 03:11 |
| sean-k-mooney | it would if you had a new enough qemu | 03:11 |
| sean-k-mooney | the underlying vulnerbality was in qemu, openstakc in genrela did not have any protection to prevetn unsafe images being passed to qemu-img | 03:12 |
| sean-k-mooney | if you have a qemu that is unpatched and supprots external datafiles | 03:12 |
| sean-k-mooney | adn you use that with a nova/glance/cinder that is unpatched then your cloud is vulnerable | 03:13 |
| sean-k-mooney | V and previous versoin are end of life or unmainteded | 03:13 |
| sean-k-mooney | so the core team does not backport CVE fixes to them regardless of how severe they are | 03:14 |
| songwenping | i find the bug cherry-picked to W, W is also in unmaintained. | 03:15 |
| sean-k-mooney | yep, there is a seprate unmained branch team | 03:16 |
| sean-k-mooney | although im not sure that was cherry picked usptram to wallaby | 03:18 |
| sean-k-mooney | it was done downstream in some distors to older branches | 03:18 |
| songwenping | where i can find the version correspondence of nova and qemu? | 03:19 |
| sean-k-mooney | https://docs.openstack.org/nova/latest/reference/libvirt-distro-support-matrix.html#min-libvirt-qemu-version-and-next-min-libvirt-qemu-version-table | 03:20 |
| sean-k-mooney | V requried qemu 4.0.0 and libvirt 5.0.0 | 03:20 |
| songwenping | our product use qemu 4.2.0 and libvirt 6.6.0 , but cannot reproduce this bug. | 03:24 |
| sean-k-mooney | it was added in qemu 4.0.0 https://github.com/qemu/qemu/commit/65a3d073e9766ed878474525118dbf0c7233caab | 03:25 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 11:14 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [doc]Developer doc about PCI and SRIOV testing https://review.opendev.org/c/openstack/nova/+/928834 | 11:14 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor obj_make_compatible to reduce complexity https://review.opendev.org/c/openstack/nova/+/928590 | 11:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456 | 11:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 11:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [doc]Developer doc about PCI and SRIOV testing https://review.opendev.org/c/openstack/nova/+/928834 | 11:15 |
| gibi | tkajinam: I respined the igb series based on you comments ^^ | 11:18 |
| sean-k-mooney | i didnt get to it last week but i can try kiciking it off in the background today | 11:21 |
| gibi | sean-k-mooney: thanks | 11:26 |
| tkajinam | gibi, thanks ! | 12:30 |
| bauzas | nova sessions are starting now | 13:02 |
| bauzas | sean-k-mooney: gibi: joining ? | 13:02 |
| sean-k-mooney | sure but i have to be late or it would not be me :) | 13:03 |
| dansmith | bauzas: it looks like the move of the imagebackend topic might have landed on top of the equivalent glance topic | 13:36 |
| dansmith | in 30m I need to do that in glance | 13:36 |
| dansmith | and I don't want to miss eventlet either of course ;) | 13:36 |
| bauzas | hah | 13:37 |
| bauzas | today was around both topics :) | 13:37 |
| dansmith | also image encryption is after image formats, which melwitt and I should both be in, until 1530 | 13:37 |
| dansmith | glance is very specific about its scheduling, which is nice, since other people are coming to these topics, | 13:37 |
| bauzas | image encryption topic on glance sessions ? | 13:37 |
| dansmith | with nova not strict it's hard to know, but I assume we'll go over both | 13:38 |
| dansmith | yes, for glance-cinder-and-nova encryption stuffm | 13:38 |
| dansmith | and since we asked them to redefine their stuff for us, and the image format stuff will impact, we should be there | 13:38 |
| bauzas | so, to understand correctly, you and mel wouldn't be able to be in the nova sessions before 1530UTC, right ? | 13:39 |
| dansmith | yeah | 13:39 |
| bauzas | given we have cinder-nova x-p session at 1600UTC, that basically means you both are off nova that day | 13:39 |
| dansmith | oh, I didn't see a time for the cinder one | 13:40 |
| dansmith | ugh | 13:40 |
| bauzas | no worries, I understand but then I need to find proper time for discussing the image backedn thinb | 13:40 |
| bauzas | I'll try to see if I can advance the cinder x-p session | 13:42 |
| dansmith | that would be good | 13:42 |
| opendevreview | Mina Shahsavan pour proposed openstack/nova master: nova-api: Prevent os-resetState API action on instances in disallowed VM states. https://review.opendev.org/c/openstack/nova/+/933311 | 13:43 |
| bauzas | dansmith: the cinder-nova x-p session will be moved to 1400-1430-ish | 13:48 |
| dansmith | \o/ | 13:48 |
| dansmith | as long as we have rosmaita in my session at 1400 since we need him :) | 13:48 |
| bauzas | I propose to keep the eventlet discussion to be discussed at 1530 (should be quick) | 13:48 |
| bauzas | this is just a wrap-up confirming what we need to do | 13:49 |
| bauzas | and then imagebackend discussion for ending the day | 13:49 |
| bauzas | zigo: we may discuss your py3.13 topic today at the PTG, interested in joining ? | 14:21 |
| bauzas | eek, got a network issue apparently | 14:52 |
| johnthetubaguy | @bauzas: I had the same, but it recovered just now. | 15:00 |
| bauzas | dditto | 15:02 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use native DictOpt for disk_cachemode https://review.opendev.org/c/openstack/nova/+/915390 | 15:08 |
| ralonsoh | sean-k-mooney, what time is tomorrow the Nova eventlet discussion? | 15:10 |
| ralonsoh | (because is tomorrow, right?) | 15:10 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Validate valid cache mode at config input layer https://review.opendev.org/c/openstack/nova/+/933327 | 15:11 |
| bauzas | ralonsoh: this is now | 15:32 |
| ralonsoh | bauzas, I'm in, thanks | 15:32 |
| bauzas | dansmith: melwitt: fyi, we're starting the eventlet discussion now | 15:32 |
| dansmith | we will be done soon I think | 15:33 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use native DictOpt for disk_cachemode https://review.opendev.org/c/openstack/nova/+/915390 | 15:51 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Validate valid cache mode at config input layer https://review.opendev.org/c/openstack/nova/+/933327 | 15:51 |
| bauzas | gouthamr: we should be discussing the manila virtiofs topic tomorrow around ~2pm UTC, wfy ? | 16:57 |
| bauzas | if this is just a question about where we are, we can even reply async | 16:57 |
| bauzas | tl;dr: it requires reviews from myself | 16:58 |
| opendevreview | Merged openstack/nova master: libvirt: Deprecate volume driver for unsupported volume backends https://review.opendev.org/c/openstack/nova/+/932247 | 17:13 |
| tkajinam | I posted this in oslo channel accidentally but I have another oslo.utils patch to be used by nova so I'll create a release once it's merged https://review.opendev.org/c/openstack/oslo.utils/+/932335 | 17:38 |
| tkajinam | As we discussed a lot about eventlet removal does anyone care to give quick review to https://review.opendev.org/c/openstack/nova/+/930726 which removes workaround to ancient eventlet versions ? | 17:44 |
| opendevreview | Mengyang Zhang proposed openstack/nova-specs master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova-specs/+/932653 | 17:45 |
| dansmith | tkajinam: sean-k-mooney should probably look at that | 17:58 |
| sean-k-mooney | i dont recall if that is one of the things i was trying to clean up. | 18:07 |
| sean-k-mooney | glancing at it if the bug in eventlet is already fixed then it soudn like an easy win | 18:07 |
| sean-k-mooney | ill add myslef as a reviewer | 18:08 |
| frickler | fyi, mentioned this in the oslo channel earlier: bumping to latest eventlet seems to break a couple of projects, including nova https://review.opendev.org/c/openstack/requirements/+/933257 | 18:11 |
| frickler | hberaud mentioned a change that makes read/write calls actually non-blocking, which might trigger stuff | 18:12 |
| frickler | https://github.com/eventlet/eventlet/issues/973 | 18:13 |
| sean-k-mooney | os.read is ment to block but yeild execution fo the current greenthread | 18:13 |
| JayF | that's going to be sneaky breaking for Ironic/IPA as well, I'll note | 18:14 |
| JayF | likely would cause issues with image streaming error handling (timeouts/disk failures) | 18:14 |
| JayF | stuff I wouldn't 100% expect to show up in happy path testing; but def. not unit tests | 18:15 |
| JayF | oh, thought this was -requirements, either way, info is good :D | 18:15 |
| sean-k-mooney | i have nto actully looked at the change in detail but reads and write need to be blocking and yeild | 18:15 |
| frickler | I completely do not rule out the possibility that this is actually a bug in eventlet | 18:17 |
| dansmith | um, is this a regression in eventlet? I was thinking it only monkeypatched the network IO bits | 18:17 |
| sean-k-mooney | it monkeypatches file io too | 18:17 |
| dansmith | like, is this change actually not part of the original intent of eventlet? | 18:17 |
| sean-k-mooney | eventlet shoudl not make any standard lib fucntion that is normally blocking non blocking form the point of view of the caller fo the function | 18:18 |
| frickler | best discuss in the oslo channel then where herve is actually around | 18:18 |
| dansmith | right, and I think the change doesn't actually change the behavior, it just adds trampoline calls in between non-blocking reads | 18:19 |
| sean-k-mooney | it may be doing somethign valid yes | 18:19 |
| sean-k-mooney | frickler: do you have a like to the nova failure? | 18:19 |
| sean-k-mooney | oh i see them | 18:20 |
| frickler | sean-k-mooney: in the reqs patch I linked, both py311 and functional failing | 18:20 |
| dansmith | sean-k-mooney: I don't see that it patches os.open, read, write: https://eventlet.readthedocs.io/en/latest/patching.html | 18:21 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L52 | 18:22 |
| JayF | https://github.com/eventlet/eventlet/blob/master/eventlet/green/os.py#L22 | 18:22 |
| dansmith | okay I guess I'm not sure that necessarily means it patches open() | 18:23 |
| sean-k-mooney | os is one of the one it default patches | 18:23 |
| JayF | it patches fdopen(), read(), write() and others outta tha library https://github.com/eventlet/eventlet/blob/master/eventlet/patcher.py#L533 | 18:23 |
| dansmith | I say that because mocking open is different than other things and I thought it has (for quite a while) been a C function | 18:23 |
| JayF | Yeah, eventlet overrides more than one C function and replaces it with a python-native version | 18:24 |
| sean-k-mooney | by the way the unit and functional test patches dont obviously look like eventlet falirues | 18:24 |
| JayF | that's how the major 3.12 blocker was resolved | 18:24 |
| dansmith | I know it does, I just didn't think open() itself | 18:24 |
| sean-k-mooney | https://github.com/eventlet/eventlet/blob/master/eventlet/green/builtin.py | 18:25 |
| JayF | docstring says: Wrap os.open; This behaves identically, but collaborates with the hub's notify_opened protocol. ... which implies we're both right-ish? | 18:25 |
| JayF | patched but no significant behavior change | 18:26 |
| sean-k-mooney | ya it looks more or less the same | 18:26 |
| sean-k-mooney | its just setting up a notifyier on the file descriptor | 18:26 |
| sean-k-mooney | to know when the file is ready i guess | 18:26 |
| JayF | sean-k-mooney: my knowledge of eventlet mostly stops at the patcher module :D hehe | 18:27 |
| dansmith | idk, ages ago (like icehouse timeframe) when we were transitioning from the mysql C client to the python-native one, I thought that we encountered a situation where local file i/o was not patched and that surprised me, and some of the elders explained that it wasn't in scope | 18:27 |
| sean-k-mooney | i mean we can disabel patchign of this | 18:28 |
| dansmith | but that was a long time ago and we've shuffled our own monkey patching a lot since then, so maybe we weren't patching os at the time, or maybe I'm just misremembering | 18:28 |
| sean-k-mooney | we coudl set os=false adn builtins=false | 18:28 |
| sean-k-mooney | we disabel os on windows for posix reasons | 18:29 |
| sean-k-mooney | so i can push a patch and see what that does but i need to look at the tempest job | 18:29 |
| sean-k-mooney | the unit/fucniton test failrues dont really tell us a lot about what the actual effect of the chagne is | 18:29 |
| sean-k-mooney | ok devstack only deploy nova-api neutron failed to start so we didnt finish stacking | 18:30 |
| sean-k-mooney | nova api seams to be up | 18:31 |
| sean-k-mooney | neutron.plugins.ml2.ovo_rpc RuntimeError: cannot notify on un-acquired lock | 18:31 |
| sean-k-mooney | that is the neutron server error | 18:32 |
| sean-k-mooney | oh thats coming form oslo messaging | 18:32 |
| sean-k-mooney | https://5ce7fa2fbddee5b97128-6f49bdea8e90fe6675f1ae011c4f0f22.ssl.cf5.rackcdn.com/933257/1/check/tempest-full-py3/018987a/controller/logs/screen-q-svc.txt | 18:32 |
| sean-k-mooney | frickler: i think its better to phase this as the new eventlet release is breaking oslo adn that is breakign everything else | 18:36 |
| sean-k-mooney | the nova unit/funtional test failure might eb related or flaky tests so i woudl try at least one recheck to confirm that | 18:36 |
| sean-k-mooney | frickler: the fact the devstack functional job worked is also odd | 18:39 |
| dansmith | JayF: sean-k-mooney: https://github.com/openstack/nova/blob/icehouse-eol/nova/cmd/__init__.py | 18:40 |
| dansmith | os=False | 18:40 |
| sean-k-mooney | rigth so we still do that for winwos | 18:40 |
| sean-k-mooney | *windows | 18:40 |
| dansmith | but that was for every binary on every platform, AFAIK | 18:40 |
| JayF | I think that was required to be added some point around t? u? v? era-ish for greendns to behave properly? My memory is very lossy but I remember having to back os=False outta IPA at some point | 18:41 |
| dansmith | ...at the time | 18:41 |
| sean-k-mooney | it was yes | 18:41 |
| sean-k-mooney | im trying to see when we changed it | 18:41 |
| sean-k-mooney | we sould be able to go back but i am not sure its breaking nova | 18:41 |
| dansmith | it was that way through rocky at least | 18:42 |
| dansmith | I lose the thread when the refactor to monkey_patch.py happened | 18:42 |
| dansmith | anyway, | 18:42 |
| dansmith | I'm not saying we should, I'm defending why I thought the os.open thing wouldn't affect us :) | 18:43 |
| dansmith | and proving that I'm old | 18:43 |
| sean-k-mooney | dansmith: the sdk devstack tests passed https://5355e5989bc73779bf10-cc682349d9014dd293929a9a1be5247d.ssl.cf5.rackcdn.com/933257/1/check/openstacksdk-functional-devstack/fad2996/testr_results.html | 18:43 |
| sean-k-mooney | dansmith: https://github.com/openstack/nova/commit/4f2d4d03f3a1a0a277dd04ec2abcbee2d4018fa2 | 18:46 |
| sean-k-mooney | so there was no reason given why it orginally was false and also no reason given why we shoudl remov it | 18:47 |
| dansmith | yeah :) | 18:47 |
| sean-k-mooney | dansmith: by the way i also remember os=false for what its worth | 18:47 |
| sean-k-mooney | as in that what i expected to see on master | 18:48 |
| sean-k-mooney | for now i just recheck the requiremnt patch to see if things are failing the same way consitently | 18:48 |
| sean-k-mooney | hum we changed in stein | 18:49 |
| opendevreview | Ghanshyam proposed openstack/osc-placement master: Update python classifier in setup.cfg https://review.opendev.org/c/openstack/osc-placement/+/931113 | 18:58 |
| sean-k-mooney | frickler: i was able to repoduce the nova.tests.unit.test_service.TestWSGIServer.test_server_pool_waitall failure locally | 19:12 |
| sean-k-mooney | nova.tests.unit.test_service.TestWSGIService has two failing unit tests for me too because we are not stubing calsl to bind but that happened with our without the new eventlet | 19:13 |
| frickler | sean-k-mooney: maybe the results also vary between python versions/distros. the cross jobs run with py311 on bookworm | 19:24 |
| sean-k-mooney | im using debian testing on 3.12 | 19:28 |
| sean-k-mooney | frickler: the unit test fialure is because its too specific | 19:28 |
| sean-k-mooney | the unit test is ment to assert that waitall is called when we shutdown the wsgi server | 19:29 |
| sean-k-mooney | but its asserting its called once and now its called twice | 19:29 |
| sean-k-mooney | so the fix for that is just change the assert | 19:29 |
| sean-k-mooney | im repoducing the functional test failure now | 19:29 |
| sean-k-mooney | the functionatl test that is failing is restarting the scheduler and that reulst in self.service_ref being None | 19:40 |
| sean-k-mooney | i think this is an artifact of how we are doing that | 19:40 |
| sean-k-mooney | restarting services in our functional tests is kind of a hack | 19:41 |
| sean-k-mooney | so again i think this will be a case of a slight test tweak but not an actual regression | 19:42 |
| opendevreview | sean mooney proposed openstack/nova master: [eventlet] update nova tests for eventlet 0.37.0 https://review.opendev.org/c/openstack/nova/+/933365 | 19:47 |
| sean-k-mooney | frickler: ^ i think that will do it | 19:47 |
| sean-k-mooney | i dont think that matrially changes the intent of any of those tests so if that passes ci we can merge it and at least get those jobs passign with eventlet 0.37.0 | 19:49 |
| sean-k-mooney | i can try installing that in a devstack tomorrow or next week | 19:49 |
| sean-k-mooney | assumign the requirements patch does not pass tempest-py3 this time | 19:49 |
| sean-k-mooney | ok im going to call it a day chat to folks tomorrow o/ | 19:52 |
| gouthamr | bauzas: delayed response wrt your question regarding the virtiofs topic.. there's a conflict tomorrow at 1400 UTC tomorrow, and there are TC sessions beyond that.. I don't mind us chatting about this async / over email.. I mainly wanted to understand where it falls in the team's priorities and what we/anyone can do to help | 20:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!