Storj Share GUI and CLI by default write the log(s) of each node to C:\Users\USER\.config\storjshare\logs
in windows and ~/.config/storjshare/logs
for linux. A log is generated for every 24h of operation of the node, after 24h a new log is generated with a new timestamp.
These logs contain valuable information regarding the traffic generated during operation of the node, this includes the amount of downloads, the amount of paid and unpaid uploads (mirror traffic). Mirror upload traffic is free because shards are transferred between farmers instead of to a paying renter. Even though its not paid, mirror traffic is essential for the survival of the network, it is a common interest of farmers to fulfill mirror requests, without it the network would not be able to survive.
Now make sure you have "loggerVerbosity"
set to 3
in the node's config file, this can be checked by clicking on the gear next to the node and selecting edit
(for StorjShare GUI):
...
// Determines how much detail is shown in the log:
// 4 - DEBUG | 3 - INFO | 2 - WARN | 1 - ERROR | 0 - SILENT
"loggerVerbosity": 3,
...
Running a simple script will allow a Storj farmer to check these parameters.
Note: If we delete the log files regularly the output data will be incomplete and will only take the time interval for which there are logs present. Don't forget to change the analyses time interval in the script!
Windows script
Optional: Powershell can be updated or installed for any OS: https://github.com/powershell/powershell#get-powershell
To execute the script please follow the steps below;
- Open powershell (
Windows key + R
, then type inpowershell
and hit enter). - copy the script below and paste it into the powershell window.
- change
2017-10
to the month in question, e.g.2017-11
to analyse all nodes for the month of November. - Hit enter.
Note: The logs can be placed in another folder as the user desires, however, the path in step two will then be different.
- Powershell (GigaBytes)
$mirrors=0; $shards=0; $received=0; (sls "(Mirror|Shard) (download|upload) completed.*size (\d*).*2017-10" ~/.config/storjshare/logs/*_*.log).Matches | ?{$_} | %{$s1 = $_.Groups[1].Value; $s2 = $_.Groups[2].Value; $s3 = $_.Groups[3].Value; if ($s1 -eq "Mirror" -and $s2 -eq "download") {$mirrors += $s3}; if ($s1 -eq "Shard" -and $s2 -eq "download") {$shards += $s3}; if ($s2 -eq "upload") {$received += $s3}}; Write-Host "Mirrors downloads:" ([Math]::Round($mirrors/1000/1000/1000)) "GB"; Write-Host "Shards downloads:" ([Math]::Round($shards/1000/1000/1000)) "GB"; Write-Host "Shards uploads:" ([Math]::Round($received/1000/1000/1000)) "GB";
Powershell should now output the following lines:
Mirrors downloads: ... GB
Shards downloads: ... GB
Shards uploads: ... GB
- Powershell (Bytes)
$mirrors=0; $shards=0; $received=0; (sls "(Mirror|Shard) (download|upload) completed.*size (\d*).*2017-10" ~/.config/storjshare/logs/*_*.log).Matches | ?{$_} | %{$s1 = $_.Groups[1].Value; $s2 = $_.Groups[2].Value; $s3 = $_.Groups[3].Value; if ($s1 -eq "Mirror" -and $s2 -eq "download") {$mirrors += $s3}; if ($s1 -eq "Shard" -and $s2 -eq "download") {$shards += $s3}; if ($s2 -eq "upload") {$received += $s3}}; Write-Host "Mirrors downloads:" $mirrors; Write-Host "Shards downloads:" $shards; Write-Host "Shards uploads:" $received;
Powershell should now output the following lines:
Mirrors downloads: ...
Shards downloads: ...
Shards uploads: ...
Mirrors downloads - Shards that have been downloaded by other nodes (the transfer is not paid, but the storage is, it is included in the GB calculation), GB (or Bytes)
Shards downloads - Shards downloaded by renters (paid transfer), GB (or Bytes)
Shards uploads - Shards uploaded to your node(s) by the renter, the transfer is not paid but the storage is (included in GBh calculation), GB (or Bytes)
Linux script
To execute the script please follow the steps below;
- Open a terminal (bash shell).
- copy the script below and paste it into the terminal window.
- change
2018-01
to the month in question, e.g.2017-11
to analyse all nodes for the month of November. - Hit enter.
Note: The logs can be placed in another folder as the user desires, however, the path in step two will then be different.
- bash (GigaBytes)
grep -iE '(Mirror|Shard) (download|upload) completed.*2018-01' ~/.config/storjshare/logs/*_*.log | awk 'BEGIN {FS="[\" ]"} /Mirror download completed/ {kk += $14} /Shard download completed/ {sd += $14} /Shard upload completed/ {su += $14} END {printf "Mirrors downloads: %d GB\n", kk/1000^3; printf "Shards downloads: %d GB\n", sd/1000^3; printf "Shards uploads: %i GB\n", su/1000^3}'
The terminal should now output the following lines:
Mirrors downloads: ... GB
Shards downloads: ... GB
Shards uploads: ... GB
- bash (Bytes)
grep -iE '(Mirror|Shard) (download|upload) completed.*2018-01' ~/.config/storjshare/logs/*_*.log | awk 'BEGIN {FS="[\" ]"} /Mirror download completed/ {kk += $14} /Shard download completed/ {sd += $14} /Shard upload completed/ {su += $14} END {print "Mirrors downloads: " kk; print "Shards downloads: " sd; print "Shards uploads: " su}'
The terminal should now output the following lines:
Mirrors downloads: ...
Shards downloads: ...
Shards uploads: ...
Mirrors downloads - Shards that have been downloaded by other nodes (the transfer is not paid, but the storage is, it is included in the GB calculation), GB (or Bytes)
Shards downloads - Shards downloaded by renters (paid transfer), GB (or Bytes)
Shards uploads - Shards uploaded to your node(s) by the renter, the transfer is not paid but the storage is (included in GBh calculation), GB (or Bytes)
Some remarks
This script is especially useful for monitoring the discrepancy between paid and unpaid uploads. Furthermore it allows for comparison of the results with the payout sheet posted by Storj once a month on the community rocketchat.
Extracting useful information from logs
You can extract the useful info from logs on daily basis. In our examples we will use 9AM.
Windows Task Scheduler
Open powershell.exe
and execute:
$ta = New-ScheduledTaskAction -Execute "powershell.exe" -Argument "(sls '(Shard|Mirror) (download|upload) completed' ~/.config/storjshare/logs/*_*.log).Line >> ~/.config/storjshare/download.log; rm ~/.config/storjshare/logs/*_*.log"
$tt = New-ScheduledTaskTrigger -Daily -At 9am
Register-ScheduledTask -TaskName "Collect logs" -Action $ta -Trigger $tt
Linux cron
Open bash and execute:
echo "9 * * * * grep -ihE '(Shard|Mirror) (download|upload) completed' ~/.config/storjshare/logs/*_*.log >> ~/.config/storjshare/download.log && rm ~/.config/storjshare/logs/*_*.log" | crontab -
Comments
0 comments
Please sign in to leave a comment.