Author: Site Editor Publish Time: 2024-10-17 Origin: Site
The power of telecom site is considered the blood of a telecom network, while the battery is regarded as its blood reservoir, safeguarding the smooth operation of the network. However, battery maintenance has always been a challenging aspect. With manufacturers continuously lowering prices after centralized procurement, the quality of batteries has significantly declined. Every year, more than 70% of telecom power system failures are attributed to battery issues, making battery maintenance a headache for maintenance personnel. This article offers an analysis of the main causes of battery failure, which may serve as a useful reference for others.
1. On-Site Power Equipment Overview
The on-site power equipment consists of two 40kVA UPS units from a well-known international brand. The batteries were installed in 2016. Below is detailed information:
UPS Information | Battery Information |
Brand & Model: International Brand UPS UL33 | Brand & Model: 12V 100Ah |
Configuration: 40 kVA, 2 units in a parallel system, each with a load of approximately 5 kW | Number of batteries: 30 cells per group, 2 groups, totaling 60 cells |
Commissioning Date: 2006 (10 years of service) | Commissioning Date: 2016 (5 years of service) |
On June 6th, the UPS manufacturer performed routine maintenance, replacing AC and DC capacitors (5 years of service) and the fans. During the battery discharge testing (20 minutes), it was found that the battery's discharge performance was poor. The discharge current was 16A, and after 10 minutes of discharge, the voltage of several cells dropped to 11.6V, but no bulging of the batteries was observed.
It was found that both UPS battery groups had bulging issues during the inspection. Using a multimeter, they measured the battery charging ripple voltage (measured using the AC setting), which was as high as 7V (far exceeding the maintenance standard). As a result, they initially suspected that the DC filter capacitors replaced by the UPS manufacturer's engineers were faulty, causing excessive ripple voltage on the UPS's DC bus, leading to the battery bulging.
2. On-Site Failure Situation
On July 22, the team from the Research Institute conducted a safety inspection at a branch office. They discovered that the batteries of the UPS systems on the 5th floor of a building were severely bulging. If there were power outages from the grid, it was feared that the batteries might not discharge properly, potentially leading to an accident. As a result, they immediately recommended that the branch's maintenance personnel contact the manufacturer’s engineers to arrange a joint on-site investigation and troubleshooting session with all three parties the following afternoon.
Bulging of 12V Batteries
On the afternoon of July 23, the three parties arrived on-site. Upon inspection, both UPS units were found to be functioning normally, with a float voltage of approximately 404V for the batteries (in line with the set parameters). The manufacturer's engineers used a Fluke 287C multimeter (high accuracy) to measure the battery charging ripple voltage, which was approximately 0.439V. A Fluke 376 clamp meter (lower accuracy) measured around 0.4V. The results from both instruments were similar and fell within the typical ripple voltage range for the equipment (generally less than 1% of the bus voltage). This indicated that the replaced DC capacitors were compliant and functioning normally. Therefore, the previously suspected theory that the capacitor replacement caused excessive ripple voltage and the battery bulging was ruled out.
Multimeter: 0.439V
Clamp meter: approximately 0.4V
A review of the UPS system's historical records showed that, on June 6, both UPS units had undergone a 15-minute battery discharge test. After restoring the main power switch, a 6-minute equalized charging was performed, followed by a 14-minute battery discharge test by the manufacturer's engineers. After the test, the UPS system automatically initiated four consecutive 12-hour equalized charges, with each phase separated by a 1-minute interval, concluding at 5:32 a.m. on June 9. Since then, the batteries have remained in float charge mode.
Further examination of the original UPS battery settings revealed the following:
The battery life was set to 48 months (4 years), though the actual life expectancy of a 12V battery should be 5 years.
Equalized charging was set to "enabled."
The charge current limit was set to 10A.
The trigger for switching to equalized charging was set to 1A (the system would automatically switch to equalized charging if the float charge current exceeded 1A, even though the default value for this model is 0.03C10~0.05C10, meaning that the equalized charging is triggered when the float charge current reaches 3-5A. However, for unknown reasons, the manufacturer's maintenance personnel had adjusted this to 0.01C10, meaning equalized charging would be triggered when the float charge current reaches 1A).
Equalized charging protection time was set to 720 minutes (equalized charging would stop automatically after 12 hours).
3. Analysis of the Failure Causes
Based on the above circumstances, the failure process can be analyzed as follows:
The two battery groups of this UPS system had been in use for 4 years (the service life of 12V batteries is 5 years), and the battery capacity had significantly decreased. However, before the failure, the battery's external appearance was normal, with no signs of bulging. A further review of the UPS historical records from January 30, 2019 (records prior to this date were cleared) to June 6, 2020, showed that the UPS system had performed 12 equalized charging, with the longest duration being no more than 15 minutes. This indicates that the equalized charging duration set in the UPS system before maintenance was relatively short, only 15 minutes, and the UPS system's short-term equalized charging will not cause the batteries to bulge.
After maintenance and capacitor replacement, the UPS system was restarted. The control logic identified the battery as newly connected, so it initiated a 6-minute equalized charging and then switched to a float charge. However, after a subsequent 14-minute discharge test, the UPS system automatically started equalized charging to fully recharge the batteries. Due to the batteries being in use for 4 years, their internal charge retention capability had deteriorated, causing the float charge current to exceed 1A, triggering the 1A equalized charging threshold set in the UPS system (the default value for this model is 3~5A float charge current to trigger equalized charging, but for some reason, the maintenance personnel had modified this to 1A). This resulted in the UPS system repeatedly initiating the equalized charging until an internal battery open circuit finally stopped it (Otherwise, the UPS system would have continued repeated equalized charging, which could have led to the battery group catching fire). During this period, the batteries underwent four continuous equalized charging cycles over 48 hours (each cycle paused for only 1 minute every 12 hours before resuming equalized charging). After such prolonged equalized charging, the batteries eventually developed bulging, and even the venting valves became deformed.
4. Conclusion
Based on the above observations and analysis, the causes of the battery failure in this UPS system are as follows:
The direct cause was the improper setting of the UPS system’s charging parameters, which led to continuous equalized charging for 48 hours with only 1-minute intervals between each cycle. Even new batteries would not withstand such prolonged and intense equalized charging, leading to the battery bulging failure in this case.
The UPS system model is an early design with functional limitations. This older UPS model (designed 20 years ago) lacked the " equalized charging interval protection time" setting (other brands typically set this interval to 7 days), resulting in continuous multiple equalized charging cycles.
The batteries' performance had degraded due to age (4 years in service), with reduced discharge capacity and poor charge retention. Before June 6, the equalized-to-float-charge conversion current threshold was set unreasonably low (only 1A for 100Ah batteries). The UPS system's default value is 3~5A, yet the maintenance personnel inexplicably modified it to 1A.
The UPS system had been in operation for 14 years, well beyond its decommissioning age, making measurement errors unavoidable. These errors may have caused the system to repeatedly initiate equalized charging due to inaccurate current detection.
Fortunately, an open circuit in one of the battery cells prevented the UPS system from continuing the repeated equalized charging cycles after the fourth equalized charging, thus avoiding the potential for the batteries to catch fire.
5. Remedial Measures for the Failure
The remedial measures include two aspects:
First, temporarily modify the UPS battery charging parameters:
Disable the equalized charging setting in the UPS system.
Adjust the trigger current for switching from float charge to equalized charging to 3A (though 3A is still somewhat low, as the default minimum is 3A, but it was previously set to 1A).
Adjust the equalized charging protection time to 1 hour (previously set to 12 hours).
Second, the branch office replaced the two battery groups with backup batteries, but the backup batteries have a capacity of only 50 Ah, so they can only be used for temporary emergency purposes. There are plans to transfer the load from the UPS system to other power sources in the future, to thoroughly resolve the power supply safety issues.
The operator spends a considerable amount annually on maintenance services for the UPS system, yet due to the negligence and carelessness of the maintenance personnel, they even mistakenly modified the default values of the UPS system, which is truly unbelievable. It is recommended that the UPS manufacturer take the maintenance of their products seriously and avoid making such basic mistakes in the future, improving the quality of their maintenance services. Meanwhile, it is suggested that the operator also pay closer attention to the subsequent maintenance services provided by the UPS manufacturer and establish an evaluation system to continuously improve the safe and reliable operation of the UPS system.
