The upload attribute name has been changed from 'user.sunnydv.upload' to 'user.sunny.upload' to improve consistency in nomenclature. Furthermore, logic has been added to handle case when there is no network connection available; the process will now sleep and retry rather than crashing or hanging.
The sunnylink_uploader.py file was modified to include headers in the upload request. The headers are gathered from the 'headers' field in the URL response JSON and passed into the 'put' request method. The debug log message was also updated to display these headers along with the upload URL.
The commit introduces more detailed log events when an attempted upload fails for any reason. Now, when there is a non-null status code like 401 or 403, it will log more information, including the decoded content of the HTTP response. This aids in quickly identifying and resolving upload issues.
Updated the base URL for SUNNYLINK to point to a new staging test environment. Changed the UPLOAD_ATTR_NAME in the sunnylink uploader from 'user.sunny.upload' to 'user.sunnydv.upload'. This provides distinction and prevents potential conflicts.
Removed 401 and 403 status codes from the success conditions in the sunnylink_uploader.py script. The handling for these status codes has been moved to the else clause, as they were unexpectedly marked as successful previously.
The commit adds sunnylink_uploader to the process_config and implements an uploader in a new file, sunnylink_uploader.py. The uploader handles listing, prioritizing, and uploading files to Sunnylink. Modified process_config to conditionally include the uploader process based on SunnylinkEnabled param and file existence. New getxattr and setxattr functions are also used to keep track of the uploaded files.
The import of the 'os' module is eliminated to clean up the code, as it is no longer needed. Furthermore, the code for removing the extended attribute from all files in the current directory has been removed as well. This streamlines the setxattr function and focuses only on setting the attributes without additional file operations.
Adjusted the logging level for certain log entries from info to debug in athenad.py and sunnylinkd.py. This change improves control over the log verbosity allowing for cleaner log files, and to debug specific sections more effectively.
The code for calculating the size of the log file has been revised. Instead of using os.path.getsize, which simply captures the size of the file on disk, we're now calculating the size of the payload after it has been serialized and encoded, plus an overhead of 100 bytes. This change provides a more accurate measure of the data that will be sent.
The `add_log_to_queue` function has been updated to compress and base64 encode log files that exceed a certain size, specifically for the "sunnylink" scenario. This function also provides logging for various stages of the process, including initial file size, size post-compression/encoding, and whether the final payload is small enough to be sent in one request. The change will improve handle of larger log files and prevent payload size related issues.
This commit decreases the maximum chunk size from 32KB to 28KB in the add_log_to_queue function in athenad.py. This size reduction will affect how the log files are loaded and processed.
Updating the maximum chunk size constant in athenad.py to decrease its value from 128KB to 32KB. This change has been made to optimize the handling of log files and data transmission.
The changes refactor the log handling in athenad.py for more efficient processing. The log is now split into maximum chunk sizes of 128KB to prevent oversized requests. Additionally, an invalid request or response exception has been introduced for better error handling.
This commit introduces network metering and PrimeType checking in sunnylinkd. The implementation includes an additional threading event and a new log handler function, 'sunny_log_handler'. It defines PrimeType and metering conditions to set and clear the new threading event as part of the main processing loop in handle_long_poll function.
Jason Wen