| *** dmellado74 is now known as dmellado7 | 00:06 | |
| rpittau | good morning ironic! Happy new year! o/ | 08:00 |
|---|---|---|
| masghar | Good morning! | 09:30 |
| opendevreview | Damien RANNOU proposed openstack/ironic-python-agent master: [WIP] USB device discovery https://review.opendev.org/c/openstack/ironic-python-agent/+/902769 | 10:57 |
| opendevreview | Damien RANNOU proposed openstack/ironic-python-agent master: [WIP] USB device discovery https://review.opendev.org/c/openstack/ironic-python-agent/+/902769 | 13:29 |
| opendevreview | Riccardo Pittau proposed openstack/ironic master: Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic/+/903861 | 13:36 |
| opendevreview | Riccardo Pittau proposed openstack/ironic-inspector master: Handle LLDP parse Unicode error https://review.opendev.org/c/openstack/ironic-inspector/+/903760 | 13:37 |
| TheJulia | good morning | 14:10 |
| * TheJulia tries to wake up | 14:14 | |
| TheJulia | Hi Nisha_Agarwal | 14:57 |
| Nisha_Agarwal | TheJulia, hi | 14:57 |
| opendevreview | Merged openstack/ironic-python-agent master: Update to latest pep8/code style versions https://review.opendev.org/c/openstack/ironic-python-agent/+/904416 | 14:57 |
| Nisha_Agarwal | This is regarding RHOSP bug | 14:58 |
| Nisha_Agarwal | There is a comment saying that there are different usernames for different partitions... | 14:58 |
| TheJulia | Yes, I'm looking at it, and leaning towards not actually an OSP or even sushy bug | 14:58 |
| TheJulia | I'm just trying to figure out why the session is disqualified | 14:58 |
| Nisha_Agarwal | i dont know what is the meaning of that | 14:58 |
| TheJulia | but the API is all lower cased, which makes no sense | 14:58 |
| TheJulia | but also is non-conformant to dsp0266 | 14:59 |
| Nisha_Agarwal | Which API? | 14:59 |
| TheJulia | the responses coming from the BMC | 14:59 |
| TheJulia | or, 'bmc' | 14:59 |
| Nisha_Agarwal | still i didnt get | 15:00 |
| Nisha_Agarwal | can u elaborate or quote what u meant | 15:00 |
| TheJulia | uhh sure | 15:00 |
| TheJulia | ummmm | 15:00 |
| TheJulia | for example, you see the error about the "Actions" field in the logs, yes? | 15:00 |
| Nisha_Agarwal | Yes | 15:00 |
| Nisha_Agarwal | that comes because the token is invalidated befor eteh get call and gets the 401 status | 15:01 |
| TheJulia | Okay, so if you trust me on this real quick, We expect the field name to be 'Actions' | 15:01 |
| TheJulia | search the same logs for '_actions' | 15:01 |
| Nisha_Agarwal | even if the status is 401, that data is passed to the next GET call for the systems | 15:01 |
| Nisha_Agarwal | it is conformant to DMTF. | 15:02 |
| TheJulia | https://redfish.dmtf.org/schemas/v1/ComputerSystem.v1_21_0.yaml | 15:02 |
| TheJulia | as an OEM field only | 15:02 |
| TheJulia | Id is missing, which we expect as required, and 'boot' is also lower cased | 15:03 |
| TheJulia | https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.19.0.pdf <-- take a look at page 61 as well | 15:04 |
| TheJulia | Page 92, 9.6.14 | 15:06 |
| TheJulia | Page 90, 9.6.6 | 15:07 |
| TheJulia | Page 95, 9.7.1 Naming rules should provide insight | 15:08 |
| Nisha_Agarwal | Ok | 15:08 |
| TheJulia | As for why the session token is suddenly invalid, I have no clue at present. | 15:08 |
| Nisha_Agarwal | See the output from the ssytem here | 15:09 |
| Nisha_Agarwal | https://paste.openstack.org/show/bvmS7TiurFavZaHA1srg/ | 15:09 |
| TheJulia | It is not like sushy is self invalidating | 15:09 |
| Nisha_Agarwal | this is the output from GET on "https://<IP>/redfish/v1/Systems/Partition0" | 15:09 |
| TheJulia | weird | 15:10 |
| TheJulia | a raw curl correct? | 15:10 |
| Nisha_Agarwal | yes | 15:10 |
| Nisha_Agarwal | i have done the curl call | 15:10 |
| Nisha_Agarwal | and secondly if it was non-DMTF, then it wouldnt have passed the deploy when we used basic as the authentication mechanism | 15:10 |
| Nisha_Agarwal | and through devstack actually the same system works | 15:11 |
| Nisha_Agarwal | thats the most wierd thing | 15:11 |
| TheJulia | https://www.irccloud.com/pastebin/AZsJvEva/ | 15:11 |
| TheJulia | That is what sushy gets in response.... | 15:11 |
| TheJulia | could we somehow be hitting some sort of different dialect | 15:12 |
| Nisha_Agarwal | we dont hit the issue when we do using devstack | 15:12 |
| TheJulia | I looked at a another log from a recent 17.x case with full blown HPE servers, Everything was upper case as we expect | 15:13 |
| TheJulia | I guess a challenge is the logs I'm looking at have rotated at least once | 15:15 |
| TheJulia | Things start with us trying to call task.driver.management.detect_vendor(task) | 15:16 |
| TheJulia | When you did devstack, what version of devstack and what version of the sushy library? Maybe there is a difference between wallaby and the version you used | 15:18 |
| TheJulia | ? | 15:18 |
| TheJulia | Also, when you curled, did you do it as a session request, or a basic auth interaction? | 15:20 |
| TheJulia | Did session auth work on devstack? | 15:21 |
| Nisha_Agarwal | TheJulia, when we did devtsack we tested with bobcat release for this server | 15:23 |
| Nisha_Agarwal | when we used curl, that time i did it as a session token | 15:24 |
| Nisha_Agarwal | session auth token worked with devstack | 15:24 |
| TheJulia | so this comes down to something between Wallaby and Bobcat | 15:25 |
| Nisha_Agarwal | and moreover if there is DMTF issue then it will not work with basic authentication also | 15:25 |
| Nisha_Agarwal | but for it has worked with basic authentication | 15:26 |
| TheJulia | well, there *is* with the responses we get from the bmc when we attempt session auth | 15:26 |
| TheJulia | well, we authenticate, things go sideways afterwards | 15:27 |
| Nisha_Agarwal | yes authenication fails and returns 401 | 15:27 |
| Nisha_Agarwal | i did that manually | 15:27 |
| TheJulia | well, in the logs I have, it works, we get a response back, we don't log it was invalid or missing the session uri | 15:28 |
| TheJulia | the connector would then use it to retry the request | 15:28 |
| Nisha_Agarwal | with default authentication as "auto", when we keep on trying deploy upon failures , then out of 10 times it crosses that GET call one time and takes the system to wait-callback | 15:29 |
| Nisha_Agarwal | this above ^ behaviour is seen on current RHOSP 17.1 version | 15:30 |
| TheJulia | Yeah, auto's handling is only if session auth doesn't work, it does work upfront, it is just that the response behavior is exceptionally weird, i.e. all lower case responses with limited fields | 15:30 |
| Nisha_Agarwal | https://paste.openstack.org/show/bKBx58BeU1dYEyXPnJdJ/ | 15:31 |
| Nisha_Agarwal | this is the pdb log which i had collected in Dec | 15:32 |
| TheJulia | When you created your session, how did you create it? I'm wondering if maybe there is a different session service path or something going on because we did retool some of the session service handling at one poitn | 15:32 |
| Nisha_Agarwal | https://<IP>/redfish/v1/SessionService/Sessions/ | 15:32 |
| TheJulia | so was that from devstack | 15:34 |
| TheJulia | ? | 15:34 |
| Nisha_Agarwal | Strange thing is that when we just remove the header "Connection: close" it works fine in RHOSP environment | 15:34 |
| Nisha_Agarwal | the above pdb is from RHOSP sushy library | 15:34 |
| TheJulia | ok | 15:34 |
| TheJulia | That is just bizare that forcing the connection to not close would change the response body. | 15:35 |
| Nisha_Agarwal | It just doesnt change the response body as i know but it doesnt invalidates the token (i.e. it doesnt get the 401 status) | 15:36 |
| TheJulia | wait, is the BMC invalidating the token? | 15:36 |
| Nisha_Agarwal | as when we dont use the header as "Connection: close" the default connection header is "Connection: keep-alive" | 15:36 |
| Nisha_Agarwal | Looks like.... | 15:36 |
| TheJulia | Yeah, under no circumstances can we use keep-alive | 15:36 |
| TheJulia | by chance, did you try upstream sushy from the wallaby branch at all? | 15:39 |
| Nisha_Agarwal | latest sushy? | 15:41 |
| Nisha_Agarwal | upstream wallaby we have not tried in this RHOSP setup. We can try | 15:42 |
| Nisha_Agarwal | TheJulia, it looks like sushy has some mapping for _actions etc field names as deploy works fine when we use basic authentication and even when we tested with devstack | 15:43 |
| TheJulia | The internal storage name is _actions actually | 15:45 |
| TheJulia | but that only gets assigned after it logged the response, which seems more like coincidence than anything else. | 15:46 |
| TheJulia | so, rhosp 17.1 appears to have the session auth fixes we did back in 2022 | 15:46 |
| Nisha_Agarwal | https://paste.openstack.org/show/brnNPbcb2DNovLnvC8yH/ | 15:50 |
| Nisha_Agarwal | this is the output from upstream stable wallaby | 15:50 |
| TheJulia | heh | 15:52 |
| TheJulia | so it works it looks like | 15:52 |
| Nisha_Agarwal | can we delete this paste file, it has complete credentials of the system | 15:52 |
| TheJulia | nope.... | 15:52 |
| Nisha_Agarwal | i mistakenly forgot to remove it | 15:52 |
| Nisha_Agarwal | editing is possible? | 15:53 |
| TheJulia | fungi: o/ you around | 15:53 |
| fungi | i am | 15:53 |
| TheJulia | any way to nuke a paste ? | 15:54 |
| fungi | yeah, just a sec and i'll wipe it | 15:54 |
| TheJulia | much appreciated | 15:54 |
| TheJulia | okay, so... why does rhosp and wallaby behave differently | 15:54 |
| opendevreview | Damien RANNOU proposed openstack/ironic-python-agent master: [WIP] USB device discovery https://review.opendev.org/c/openstack/ironic-python-agent/+/902769 | 15:55 |
| fungi | Nisha_Agarwal: TheJulia: i've deleted that paste now, so it's no longer accessible, but you likely still want to consider anything you pasted compromised since someone may have seen it in the interim | 16:00 |
| Nisha_Agarwal | fungi thanks | 16:00 |
| TheJulia | thanks fungi! | 16:00 |
| fungi | any time! don't hesitate to let me know if you need anything else | 16:02 |
| Nisha_Agarwal | TheJulia, strange thing is that when i use the python shell inside container and do same calls, the GET call for the Systems URI gets same output as upstream wallaby | 16:09 |
| Nisha_Agarwal | https://paste.openstack.org/show/bct9awc403ymTYL7yITj/ | 16:09 |
| Nisha_Agarwal | this is from the RHOSP ironic-conductor container | 16:10 |
| Nisha_Agarwal | TheJulia, do u still need the databse dump in the bugzilla | 16:10 |
| Nisha_Agarwal | if so please share the procedure to create the ironic db dump in RHOSP | 16:11 |
| TheJulia | so that is valid output, what we would expect | 16:11 |
| TheJulia | so, No, I don't need the DB, initially I was thinking this was some weird session handling bug | 16:11 |
| Nisha_Agarwal | TheJulia, it still looks like that | 16:12 |
| Nisha_Agarwal | but the output in RHOSP is different than what we do manually is more wierd | 16:12 |
| TheJulia | yeah, can you access the manufacturer field on the system object? | 16:12 |
| Nisha_Agarwal | I do not see the manufacturer field in the output | 16:16 |
| TheJulia | hmm, what about the id field, I guess the thing is your looking at raw json the object is based upon, not the loaded object output | 16:18 |
| Nisha_Agarwal | "Id": "Partition0", | 16:20 |
| rpittau | good night! o/ | 17:15 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Handle Power On/Off for child node cases https://review.opendev.org/c/openstack/ironic/+/896570 | 22:47 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Redfish UefiHttp boot support https://review.opendev.org/c/openstack/ironic/+/900964 | 23:03 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Add HTTP versions of network boot interfaces https://review.opendev.org/c/openstack/ironic/+/900965 | 23:03 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Log upon completion of power sync https://review.opendev.org/c/openstack/ironic/+/891334 | 23:06 |
| opendevreview | Julia Kreger proposed openstack/ironic master: docs: add distribution specific chain of trust warning around grub https://review.opendev.org/c/openstack/ironic/+/904597 | 23:24 |
| opendevreview | Julia Kreger proposed openstack/ironic master: Make bandit voting on check and gate https://review.opendev.org/c/openstack/ironic/+/879498 | 23:31 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!