* zstd uploader
* fix that
* fix name of function
* comment
* log failed
* fix comma_api_source for routes with both bz2 and zst rlogs
* TODO
* 10-14 achieves almost no benefit on qlogs in a few cases, but takes 2x the time
* these aren't written out
* regen: specify any list of sources
ooh this is pretty nice
* regen and process replay
* damn, actually we don't need all this (cool tho)
Revert "regen: specify any list of sources"
This reverts commit ceb0b4abed9ad463a9fe98d9b98a05875a52806f.
* just let it auto resolve
* fix athenad/uploader tests
* zst here too
* TODOs
* yes
* Revert "TODOs"
This reverts commit 8c7da1dbd0340c72290b5eb5563b642080ddc131.
* Revert "zst here too"
This reverts commit 23b0023ddfd22c8090be7a7caa09e7026a12aa5c.
* Revert "just let it auto resolve"
This reverts commit f296d62424227ad05facc62abc18a6f81b474e84.
* Revert "regen and process replay"
This reverts commit 0768330e96974a42616d229d159780619d049cd0.
* revert readme
* not in save_log either
* lfg
* Revert "lfg"
This reverts commit 3718559c6c4de7d1f0c80dc9f1a1d335fe679a89.
The upload attribute name has been changed from 'user.sunnydv.upload' to 'user.sunny.upload' to improve consistency in nomenclature. Furthermore, logic has been added to handle case when there is no network connection available; the process will now sleep and retry rather than crashing or hanging.
The sunnylink_uploader.py file was modified to include headers in the upload request. The headers are gathered from the 'headers' field in the URL response JSON and passed into the 'put' request method. The debug log message was also updated to display these headers along with the upload URL.
The commit introduces more detailed log events when an attempted upload fails for any reason. Now, when there is a non-null status code like 401 or 403, it will log more information, including the decoded content of the HTTP response. This aids in quickly identifying and resolving upload issues.
Updated the base URL for SUNNYLINK to point to a new staging test environment. Changed the UPLOAD_ATTR_NAME in the sunnylink uploader from 'user.sunny.upload' to 'user.sunnydv.upload'. This provides distinction and prevents potential conflicts.
Removed 401 and 403 status codes from the success conditions in the sunnylink_uploader.py script. The handling for these status codes has been moved to the else clause, as they were unexpectedly marked as successful previously.
The commit adds sunnylink_uploader to the process_config and implements an uploader in a new file, sunnylink_uploader.py. The uploader handles listing, prioritizing, and uploading files to Sunnylink. Modified process_config to conditionally include the uploader process based on SunnylinkEnabled param and file existence. New getxattr and setxattr functions are also used to keep track of the uploaded files.
The import of the 'os' module is eliminated to clean up the code, as it is no longer needed. Furthermore, the code for removing the extended attribute from all files in the current directory has been removed as well. This streamlines the setxattr function and focuses only on setting the attributes without additional file operations.
This commit adapts the xattr handling in the loggerd module to individual platforms, specifically macOS and others. It imports specific xattr functions based on the running system, enhancing compatibility and reducing the risk of errors. In specific, 'ENOATTR' error is now also taken into account which is relevant in some non-Linux platforms.
Jason Wen