Hyundai IONIQ Forum banner
1 - 20 of 44 Posts

·
Registered
Joined
·
170 Posts
Discussion Starter · #1 · (Edited)
Steps to reproduce:

1. Discharge the battery down to 34%. (possibly not necessary, I did this because I was investigating if battery saver activates under 35% SOC)
2. Wait until the battery saver feature activates. (timing may be important, it may trigger in a narrow window of time)
3. Plug the car in to charge on AC power

Outcome: The traction battery will charge on AC power WITHOUT ENABLING THE +12 DC-DC CONVERTER! This will quickly drain it dry (looks like in about 2-3 hours it will be dead)

I have 2 theories how this happened. One is :
Event Plugged In -> Disable Battery saver -> Disable +12 DC-DC Converter
Event Plugged In -> Enable +12 DC-DC Converter.

One way this could happen is there are 2 control messages hitting the CAN bus at the same time and one is masking the other. (meaning the firmware does one and ignores the other, because it has no queue and when executing a state change is 'locked out' from any further commands to the contactor board)
The other way this could happen is if they are using a state machine, and this particular combination of states and state transitions: (while battery saver, initiate charging) is improperly handled. Less experienced firmware engineers don't write explicit state machines but have an implicit one in their code with various state variables and a mess of branch statements, this could be why.

About me: I have about a decade of experience in embedded control systems, a couple years working for an automotive tier 1, and I currently work on ADAS systems for a tech giant. I am not an expert in 12 battery charging logic but I have done various related systems in the past. Typically asian OEMs ignore messages from users, they would have to find this bug themselves, but maybe the Ioniq guy or someone big can replicate this bug on youtube and get their attention.

If someone knows the Ioniq Guy or Out of Spec Review's screenames on here can they ping them?

The below pictures: the first is my log showing the smoking gun failure. The second is showing that 12.13 volts is extremely bad, i am still waiting to see if it fails or saves itself at the last moment but I think it's about to fail. And finally is a picture confirming that the car IS charging, exactly at the same time when I took these logs the car thinks it is charging.

Other users have complained of failures in a similar situation, and said they took it to the dealership and got a new lead acid battery. That is NOT THE PROBLEM and WILL NOT FIX IT. This will keep happening until there is a software patch for the actual bug.

Rectangle Slope Font Parallel Pattern




Font Gas Magenta Technology Darkness

Car Vehicle Tire Automotive lighting Automotive tire

Update: it failed to fail. Hyundai let it do damage to the 12V battery but at 9 volts it activated some kind of failsafe:
Rectangle Font Slope Parallel Pattern
 

Attachments

·
Registered
Joined
·
170 Posts
Discussion Starter · #3 ·
Any way of avoiding this error until there's a fix?
Basically no, but it won't happen often. It looks like it requires specific conditions and it could be that they are narrow and i just got 'lucky' this time. (for instance it might only happen if the CAN bus traffic is high or the car is reporting home via bluelink or other conditions I don't know about)

If it does happen you are hosed unless you have a battery monitor or jump pack. It could happen when you are charging at work, etc. There is no indication from the outside of the car that this failure has happened, you won't know until the battery is dead. (at 17%, waiting for it to fail so I can test out my jump pack. Pretty sure it is going to fail, hyundai isn't going to let the lead acid battery get to 15% SOC and then save it).
 

·
Registered
Joined
·
118 Posts
Basically no, but it won't happen often. It looks like it requires specific conditions and it could be that they are narrow and i just got 'lucky' this time. (for instance it might only happen if the CAN bus traffic is high or the car is reporting home via bluelink or other conditions I don't know about)

If it does happen you are hosed unless you have a battery monitor or jump pack. It could happen when you are charging at work, etc. There is no indication from the outside of the car that this failure has happened, you won't know until the battery is dead. (at 17%, waiting for it to fail so I can test out my jump pack. Pretty sure it is going to fail, hyundai isn't going to let the lead acid battery get to 15% SOC and then save it).
I will try to reproduce this behavior as soon as possible. (Now at 35% SOC and 12.6V waiting for the battery saver to activate ...)

I'm surprised that I haven't seen this happen in the 8 months I have had the car, mainly charging on AC power all the time.
If the HV-battery is charging and the +12V DC-DC converter is off for some strange reason, then what makes the 12V battery drain so fast?
There shouldn't be any 12V load, but there has to be a significant current draining the battery if you already are at 17% !

The only issue I've had with the 12V battery was the fact that the battery drained very quickly after having opened the charging port door using the BlueLink app.
This looks just like that case, so to be sure - did you by any chance use BlueLink to open the door when you plugged the car in ??
 

·
Registered
Joined
·
64 Posts
Can you test just above 35%? Possibly 36%? My follow-up question would be, can you start charging below 35%, then once it's over 35% (say 36-37%) unplug, then plug back in? Would it, at that point, begin charging the 12V?
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #8 ·
Interesting. What charger are you using? Presume it is AC ~30amp?
AC 12 amp. I don't see how this will make any difference, the issue isn't a lack of power available, it's that the battery charger is not running.
I will try to reproduce this behavior as soon as possible. (Now at 35% SOC and 12.6V waiting for the battery saver to activate ...)

I'm surprised that I haven't seen this happen in the 8 months I have had the car, mainly charging on AC power all the time.
If the HV-battery is charging and the +12V DC-DC converter is off for some strange reason, then what makes the 12V battery drain so fast?
There shouldn't be any 12V load, but there has to be a significant current draining the battery if you already are at 17% !

The only issue I've had with the 12V battery was the fact that the battery drained very quickly after having opened the charging port door using the BlueLink app.
This looks just like that case, so to be sure - did you by any chance use BlueLink to open the door when you plugged the car in ??
During HV battery charging there is a lot of load. Battery BMS. Whichever BCM manages the traction battery but hyundai may just power all the BCMs. Cooling pump. Cooling fan if necessary. All those indicator lights and their control boards. The AC->HVDC converter itself may be running on 12v from this bus! It took 4.5 hours total to drain the 12v completely flat. It's a 60amp-hour battery, so the load was 160 watts.

I used the remote to open the door.
 

·
Registered
Joined
·
4 Posts
And on the subject, is the factory battery a "deep discharge" design? I believe that an application which regularly discharges the battery to 33% of charge would make short work of a conventional "engine starting" battery. The owners' manual (Kia, to be sure, but aren't they mainly the same?) indicates a "maintenance free" battery chemistry is used, and I always thought that precluded it being a deep discharge design. I could be wrong about that.
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #13 ·
Then those of us unencumbered by "time of day" electrical service would be advised to plug in as soon as we arrive home, right?
Might help. Simply charging daily at any arbitrary time would probably keep the 12V high enough battery saver never activates, avoiding the bug. Probably similar to how Hyundai tested it which is why they didn't catch it.
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #14 ·
And on the subject, is the factory battery a "deep discharge" design? I believe that an application which regularly discharges the battery to 33% of charge would make short work of a conventional "engine starting" battery. The owners' manual (Kia, to be sure, but aren't they mainly the same?) indicates a "maintenance free" battery chemistry is used, and I always thought that precluded it being a deep discharge design. I could be wrong about that.
It's not maintenance free and it says deep cycle on the side. Appears to be flooded lead acid and when I checked the part number that appears to be the case. These can be maintained by adding distilled water and using a desulfator.
 

·
Registered
Joined
·
118 Posts
FYI: Just finished my test;

Waited until the battery saver activated, and let it charge for about 10 min.
Opened the charge port door (using the remote key), and plugged the car in to charge.
I couldn't see any changes in the voltage or even spikes at that time, and the HV-battery has now been charging the 12V battery normally about 5 hours with a constant 14,73V.
(there's no need to include a picture, because its just a strait line ...)

You are of course right that if the 12V DC-DC converter is completely dead (not only the module that controls the 12V charging), then all power for the 12V buses come from the 12V battery itself.
But 160W (13,3 amps constant current) - isn't that a lot, even if all the BCM:s are powered ...

As I said, I now started at 35% SOC, and that might be a bit too high(?)
Will do a new test starting at a lower SOC the next time I charge the car.
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #16 ·
FYI: Just finished my test;

Waited until the battery saver activated, and let it charge for about 10 min.
Opened the charge port door (using the remote key), and plugged the car in to charge.
I couldn't see any changes in the voltage or even spikes at that time, and the HV-battery has now been charging the 12V battery normally about 5 hours with a constant 14,73V.
(there's no need to include a picture, because its just a strait line ...)

You are of course right that if the 12V DC-DC converter is completely dead (not only the module that controls the 12V charging), then all power for the 12V buses come from the 12V battery itself.
But 160W (13,3 amps constant current) - isn't that a lot, even if all the BCM:s are powered ...

As I said, I now started at 35% SOC, and that might be a bit too high(?)
Will do a new test starting at a lower SOC the next time I charge the car.
Well this explains why Hyundai didn't find it. I worked at a tier 1 a couple years ago, we had bugs that were 1/1000. Seems to be an intermittent glitch.
160 watts is a lot though it may include coolant pump motors and fans. That would explain most of it. It may also be what powers the charging circuits themselves - obviously it converts to DC then high frequency AC to boost to about 800 volts. All the fets and logic that do this get their power from somewhere. Obviously most battery chargers you get power themselves but Hyundai could have saved a few parts if they get the power from the 12v bus.
 

·
Registered
Joined
·
4 Posts
It's not maintenance free and it says deep cycle on the side. Appears to be flooded lead acid and when I checked the part number that appears to be the case. These can be maintained by adding distilled water and using a desulfator.
THANKS. I always appreciate proper design. So much for the "boilerplate" owners' manual.
Well this explains why Hyundai didn't find it. I worked at a tier 1 a couple years ago, we had bugs that were 1/1000. Seems to be an intermittent glitch.
160 watts is a lot though it may include coolant pump motors and fans. That would explain most of it. It may also be what powers the charging circuits themselves - obviously it converts to DC then high frequency AC to boost to about 800 volts. All the fets and logic that do this get their power from somewhere. Obviously most battery chargers you get power themselves but Hyundai could have saved a few parts if they get the power from the 12v bus.
If I understand correctly, then under "normal" conditions the car enables the HV-12VDC converter during the charge cycle independent of the 12V battery state of charge, and necessarily so because there is a substantial 12V load during a charge cycle. The "glitch" will occur only if charging is initiated while the 12V charge cycle is active, but even then only occasionally.
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #18 · (Edited)
If I understand correctly, then under "normal" conditions the car enables the HV-12VDC converter during the charge cycle independent of the 12V battery state of charge, and necessarily so because there is a substantial 12V load during a charge cycle. The "glitch" will occur only if charging is initiated while the 12V charge cycle is active, but even then only occasionally.
Correct. This isn't the only glitch I have seen some marginal behavior at other times.

I have written code early in my career that was cantankerous like this. I pretty soon learned how to make this kind of behavior not happen, it takes the right mindset and careful control of state and so on.
 

·
Registered
Joined
·
200 Posts
Correct. This isn't the only glitch I have seen some marginal behavior at other times.

I have written code early in my career that was cantankerous like this. I pretty soon learned how to make this kind of behavior not happen, it takes the right mindset and careful control of state and so on.
I spent decades in the embedded software/firmware world and what you write regarding the problem rings true to me.
 

·
Registered
Joined
·
170 Posts
Discussion Starter · #20 ·
I spent decades in the embedded software/firmware world and what you write regarding the problem rings true to me.
Well yeah and the other thing is you don't make these bugs go away by opening up a jira, finding a problem statement in the form of "when X it does Y", open the source, slap an if statement in "if x: y=disabled", and mark it as 'fixed'.
Because that probably just made 2 new bugs you don't know about.

Though what usually happens and happened in this case is "when X it does Y", someone at Hyundai will try X, find that it "works as intended" when they tried it, and mark it as "could not reproduce".
 
1 - 20 of 44 Posts
Top