danvics/R710-Fan-Control
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Make use of smart-intercept-spindown and smartctlnvme

Browse source 
Migrate smart-intercept-spindown and smartctlnvme into this repo from
my ansible-initial-server-setup playbooks, so they can be published in
a self-contained repo.  Ansible then distributes these files anyway
via including this repo.

This enables us to get rid of the spindown test code we duplicated
here, plus ensure we're actually distributing smartctlnvme
This commit is contained in:
Tim Connors 2025-09-11 04:04:08 +10:00
parent 2b6c1590f1
commit 7c831f1f52
4 changed files with 170 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
hddtemp
View file

@ -22,11 +22,9 @@ $ENV{PATH}="$ENV{PATH}:/usr/sbin:/sbin";
my $last_exit=0;
foreach my $drive (@ARGV) {
my $sense=`sdparm --command=sense $drive 2>/dev/null`;
if (!($sense =~ /Standby/)) {
my $output=`smartctl -A -i --nocheck=standby,0 $drive`;
$last_exit = $?>>8;
my $output=`/usr/local/bin/smart-intercept-spindown -A -i $drive`;
$last_exit = $?>>8;
if (!($output =~ /Standby condition/)) {
my $model="";
if ($output =~ /(Model Number|Device Model):\s*(.*)/) {
$model="$2: ";

45
readme.md
View file

@ -34,9 +34,16 @@ I wrote it the night before Australia's hottest December day on record
coping so far now that it has reached that predicted peak (I don't
believe it's only 26 in my un-air conditioned study).
It also includes my own hddtemp implementation that intercepts
smartctl so it doesn't spin up spundown disks that aren' always
detected by `smartctl --nocheck=standby,0`, and tries to parse SAS
drives and NVME drives via a forked [munin-smart-nvme
repo](https://github.com/spacelama/munin-smart-nvme), munging them
into the same format (and subject to the same spindown tests).
# installation (debian/proxmox)
It's in my [ansible
This repo is included via my [ansible
module](https://github.com/spacelama/ansible-initial-server-setup/tree/master/roles/dell_server)
(which you won't want to use in full, but you can certainly adapt),
but manual installation is:
@ -47,7 +54,7 @@ sudo apt install liblist-moreutils-perl lm-sensors ipmitool
sudo apt remove hddtemp
sudo cp -p poweredge-fand.pl /usr/local/bin && sudo chmod 755 /usr/local/bin/poweredge-fand.pl
sudo cp -p hddtemp /usr/local/bin && sudo chmod 755 /usr/local/bin/hddtemp
sudo cp -p hddtemp smart-intercept-spindown smartctlnvme /usr/local/bin/ && sudo chmod 755 /usr/local/bin/{hddtemp,smart-intercept-spindown,smartctlnvme}
sudo cp -p poweredge-fand.service /etc/systemd/system/poweredge-fand.service
sudo systemctl daemon-reload
sudo systemctl --now enable poweredge-fand.service
@ -56,11 +63,26 @@ sudo systemctl --now enable poweredge-fand.service
[Reddit discussion](https://www.reddit.com/r/homelab/comments/ed6w7y)
# Possibly required modifications/tuning
The code's configuration is in the script, unfortunately. It's also configured for my *specific* R730XD's in my specific climate with my specific drives and tolerance for noise. This code comes with no warranty - you are expected to both tune it, and monitor for possible failures or things being too hot. For the R710, you'll probably need to modify the regexps looking for "Inlet Temp" to whatever's your version of ambient air temperature - you might need to anchor the text since it's only using grep to filter the results.
The code's configuration is in the script, unfortunately. It's also
configured for my *specific* R730XD's in my specific climate with my
specific drives and tolerance for noise. This code comes with no
warranty - you are expected to both tune it, and monitor for possible
failures or things being too hot. For the R710, you'll probably need
to modify the regexps looking for "Inlet Temp" to whatever's your
version of ambient air temperature - you might need to anchor the text
since it's only using grep to filter the results.
You might want to modify setpoints and thresholds. $demand isn't actually a percentage. That code is a mess, $static_speed_high is more or less arbitrary - the initial ramps are chosen to sort of scale from an input of 0-255 and map to 0x02 to 0x12 ($static_speed_low to $static_speed_high), which is bloody loud and fast on my machines, but are allowed to continue linearly even further all the way to 255 if necessary if the temperature ramps up to 6 million degrees.
You might want to modify setpoints and thresholds. $demand isn't
actually a percentage. That code is a mess, $static_speed_high is more
or less arbitrary - the initial ramps are chosen to sort of scale from
an input of 0-255 and map to 0x02 to 0x12 ($static_speed_low to
$static_speed_high), which is bloody loud and fast on my machines, but
are allowed to continue linearly even further all the way to 255 if
necessary if the temperature ramps up to 6 million degrees.
I found it simple to test by starting up a whole bunch of busy loops on each of the 32 cores in my machine, heating each core up to 60degC and making sure the fans ramped up high:
I found it simple to test by starting up a whole bunch of busy loops
on each of the 32 cores in my machine, heating each core up to 60degC
and making sure the fans ramped up high:
```
> grep processor /proc/cpuinfo | wc -l
64
@ -100,11 +122,12 @@ measures your room temperature) temperature are polled less frequently
than coretemps, given they don't change as rapidly and are more expensive
to read.
This script monitors the ambient air temperature (you will likely
need to modify the $ipmi_inlet_sensorname variable to find the correct
This script monitors the ambient air temperature (you will likely need
to modify the $ipmi_inlet_sensorname variable to find the correct
sensor), the hdd temperatures, the core and socket temperatures
(weighted so one core shooting up if all the others are still cold doesn't suddenly convert your machine into an airfreighter taking off -
let the heatsink do its job).
(weighted so one core shooting up if all the others are still cold
doesn't suddenly convert your machine into an airfreighter taking
off - let the heatsink do its job).
It uses setpoints and temperature ranges you can tune to your heart's
content. I use it to keep the fans low but increasing to a soft
@ -148,7 +171,9 @@ hysteresis](https://github.com/nabijaczleweli/tarta-crust/blob/master/r710_fan_c
# Historical Howto: Fallback and manually testing setting the fan speed of the Dell R610/R710
Historical notes and stuff it's partially relying on behind the scenes, and if you have problems with the code, you may find yourself digging through this:
Historical notes and stuff it's partially relying on behind the
scenes, and if you have problems with the code, you may find yourself
digging through this:
1. Enable IPMI in iDrac
2. Install ipmitool on linux, win or mac os

132
smart-intercept-spindown Executable file
View file

@ -0,0 +1,132 @@
#!/bin/bash
# For those disks like some of my SAS drives where `smartctl
# --nocheck=standby` and `hdparm -C` actually spins up the disk,
# detect whether a disk is spun down with sdparm commands, and don't
# invoke smartctl if it is. Otherwise, some of our other disks like
# our 16TB SATA drives on Dell Perc controllers give unreliable status to `sdparm
# --command=sense`, but `smartctl --nocheck=standby` does seem to
# reliably detect their spundown state anyway, so we continue to
# supply `--nocheck=standby` to cover this case.
# While we're here, we might as well try transform the mimimal output
# smartctl gives us on SAS drives into something we can parse too, so
# we can at least get temperatures for hddtemp_smartctl's use. We
# also detect nvme drives and pass off a request to smartctlnvme for
# it to give us smartctl style output on them.
final_arg="${@: -1}"
convert_sas() {
read_initial=true
read_data=false
device_model=
device_product=
local exit_code=0
while read line ; do
case "$read_initial,$read_data,$line" in
true,false,*DATA\ SECTION*)
read_initial=false
read_data=true
echo "Device Model: $device_vendor $device_product"
echo "$line"
echo "ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE"
;;
true,false,Vendor:*)
device_vendor="$( echo "$line" | sed 's/Vendor: *//' )"
;;
true,false,Product:*)
device_product="$( echo "$line" | sed 's/Product: *//' )"
;;
true*)
echo "$line"
;;
false,true,SMART\ Health\ Status:*)
if [[ "$line" == *OK ]] ; then
exit_code=0
else
exit_code=8
fi
# echo "$line"
;;
false,true,Current\ Drive\ Temperature:*)
temp="$( echo "$line" | sed 's/.*Temperature: *// ; s/ C$//' )"
;;
false,true,Drive\ Trip\ Temperature:*)
temp_threshold="$( echo "$line" | sed 's/.*Drive Trip Temperature: *// ; s/ C$//' )"
hundred_minus_temp="$( printf "%03.f" "$( echo "100 - $temp" | bc -l )" )"
echo "194 Temperature_Celsius null $hundred_minus_temp $hundred_minus_temp 000 Old_age Always - $temp (Min/Max 0/$temp_threshold)"
;;
false,true,Accumulated\ power\ on\ time,\ hours:minutes:*)
hours="$( echo "$line" | sed 's/.*hours:minutes // ; s/:.*//' )"
# minutes="$( echo "$line" | sed 's/.*hours:minutes // ; s/.*://' )"
# hours="$( echo "scale=2 ; $hours + $minutes/60" | bc -l )"
echo "9 Power_On_Hours null 100 100 000 Old_age Always - $hours"
;;
false,true,Specified\ cycle\ count\ over\ device\ lifetime:*)
start_stop_cycles_threshold="$( echo "$line" | sed 's/.*Specified cycle count over device lifetime: *//' )"
;;
false,true,Accumulated\ start-stop\ cycles:*)
start_stop_cycles="$( echo "$line" | sed 's/.*Accumulated start-stop cycles: *//' )"
hundred_minus_cycles="$( printf "%03.f" "$( echo "100 - 100*$start_stop_cycles/$start_stop_cycles_threshold" | bc -l )" )"
echo "12 Power_Cycle_Count null $hundred_minus_cycles $hundred_minus_cycles 000 Pre-fail Always - $start_stop_cycles"
;;
false,true,Specified\ load-unload\ count\ over\ device\ lifetime:*)
load_unload_cycles_threshold="$( echo "$line" | sed 's/.*Specified load-unload count over device lifetime: *//' )"
;;
false,true,Accumulated\ load-unload\ cycles:*)
load_unload_cycles="$( echo "$line" | sed 's/.*Accumulated load-unload cycles: *//' )"
hundred_minus_cycles="$( printf "%03.f" "$( echo "100 - 100*$load_unload_cycles/$load_unload_cycles_threshold" | bc -l )" )"
echo "193 Load_Cycle_Count null $hundred_minus_cycles $hundred_minus_cycles 000 Pre-fail Always - $load_unload_cycles"
;;
false,true,Elements\ in\ grown\ defect\ list:*)
defects="$( echo "$line" | sed 's/.*Elements in grown defect list: //' )"
echo "5 Reallocated_Sector_Ct null 100 100 001 Pre-fail Always - $defects"
;;
false,true,Non-medium\ error\ count:*)
errors="$( echo "$line" | sed 's/.*Non-medium error count: //' )"
echo "1 Raw_Read_Error_Rate null 100 100 001 Pre-fail Always - $errors"
;;
esac
done <<< "$1"
return $exit_code
}
if [ "1" = --version ] ; then
# allow munin smart_ check to test for support of --nocheck=standby to ensure it doesnt' then run hdparm
exec smartctl "$@"
fi
if sdparm --command=sense "$final_arg" 2>/dev/null | grep Standby; then
# exit code success rather than code 2 like smartctl by default
# Could do similar output with hdparm -C, but smartctl
# --nocheck=standby,0 correctly detects that
exit
fi
smart_output=$( smartctl --nocheck=standby,0 "$@" )
exit_code=$?
case "$smart_output" in
*Transport*protocol:*SAS*)
convert_sas "$smart_output"
convert_exit=$?
# FIXME: should work out a proper exit code precedence rule
if (( $convert_exit > $exit_code )) ; then
exit $convert_exit
fi
exit $exit_code
;;
*NVMe*Version*)
# to transform smartctl nvme output to more resemble
# traditional output so existing munin plugins can monitor it
exec /usr/local/bin/smartctlnvme "$@"
;;
*)
echo "$smart_output"
;;
esac
exit $exit_code

BIN
smartctlnvme Executable file
View file

Binary file not shown.