Clearing logs from PDM that do not rotate so update will progress

Updated
  • Updated on Feb 12, 2025
  • Published on Dec 30, 2024
  • 3 minute(s) read
  • b
  • Migration Team

Scope

On some PDMs (Running 2.5.X) there was an issue where the logs would not rotate properly filling the drive and preventing software updates from happening or PD-Alerts from completing.  The system must be cleaned up manually then install 2.5.8 or higher (other versions before 2.5.8 will still have log rotation issues)

A root password is required.  Please contact Telos for assistance with his procedure.

Enable SSH on the PDM by http://192.168.2.96/secureshell.html

Then using putty or other, ssh to the PDM using the password.txt file (must have module number from PDM to know the password.

It’s a good idea to run df -h first to confirm the file system is full.  Logs are kept in /var as shown below, size is 788M and 788M is used, 0 free

PDM-B980.133:~ root$ df -h
Filesystem      Size Used Avail Use% Mounted on
rootfs          152M 74M  71M   52%  /
/dev/root       152M 74M  71M   52%  /
devtmpfs        500M 0    500M  0%   /dev
tmpfs           500M 280K 499M  1%   /run
cgroup_root      10M 0    10M   0%   /sys/fs/cgroup
shm             500M 0    500M  0%   /dev/shm
/dev/sda2       788M 788M 0     100% /var
tmpfs            20M 16K  20M   1%   /var/tmp
PDM-B980.133:~ root$

To confirm the log is the offending file, from the prompt;

PDM-B980.133:log root$ cd /var/log
PDM-B980.133:var root$ ls -lh

System will show you (note the size of 257.log file) total 770M

-rw-r--r-- 1 root root 766M Nov  3  2016 257.log
-rw-r----- 1 root root  50K Mar  5 14:55 dmesg
-rw-r--r-- 1 root root    0 Mar  5 16:27 lastlog
-rw------- 1 root root 2.7M Nov  3  2016 messages
-rw-r--r-- 1 root root 2.1K Oct 27  2016 ntpd.log
-rw------- 1 root root    0 Mar  5 16:27 tallylog
-rw-rw-r-- 1 root utmp  68K Mar  5 16:27 wtmp

Now we can delete the file (any files in this log directory may be safely removed)

PDM-B980.133:log root$ rm 257.log

Another file that can get too large is /var/log/messages. So:

PDM-B980.133:log root$ rm messages

Then another look at the directory structure shows /dev/sda2 is down to 22M.  Note, there are other things kept in that dev/sda2 device. Most notable is the audio dump archive. If you are keeping those, your results may show more or less space available than what is shown in this example. 

PDM-B980.133:log root$ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          152M   74M   71M  52% /
/dev/root       152M   74M   71M  52% /
devtmpfs        500M     0  500M   0% /dev
tmpfs           500M  280K  499M   1% /run
cgroup_root      10M     0   10M   0% /sys/fs/cgroup
shm             500M     0  500M   0% /dev/shm
/dev/sda2       788M   22M  727M   3% /var
tmpfs            20M   16K   20M   1% /var/tmp

If free space is still not reclaimed, it may be the dump audio archive taking up the rest of the space. It too can fill up and exaust storage space. Current firmware prevents this problem from appearing again in the future. Below is the way to delete all the WAV dump recordings (ask customer first sometimes they need to keep these and you might have to use winSCP to copy to the local PC to save them)

PDM-B980.133:~ root$cd /var/opt/25-Seven/PDM/dumps/
PDM-B980.133:dumps root$rm *.wav

One last step is to remove a temp directory that gets left as part of the failed install. There is a status file that is left in that directory which will keep another install from happening. To clean this up, run the following;

PDM-B980.133:log root$rm -rf /var/lib/pkg_installer

From there, any further software updates should progress as normal.

Related articles