Add cookie & crumb to requests. Involves several changes:
- fetch cookie & crumb, obviously.
- two different cookie strategies - one seems to work better in USA, other better outside.
- yfinance auto-detects if one strategy fails, and switches to other strategy.
- cookie is stored in persistent cache folder, alongside timezones. Refetched after 24 hours.
To have this work well with multithreading (yfinance.download()) requires more changes:
- all threads share the same cookie, therefore the same session object. Requires thread-safety ...
- converted data class to a singleton with "SingletonMeta":
- the first init() call initialises data.
- but successive calls update its session object - naughty but necessary.
- thread locks to avoid deadlocks and race conditions.
If Yahoo returns intraday price data with dividend or stock-split event in future, then this broke the merge.
Fix is to discard out-of-range events.
Assumes that if user requesting intraday then they aren't interested in events.
Price repair fixes and improvement
Fixes:
- fix reconstruction mis-calibration with tiny DataFrames
- fix detecting last-active-trading-interval when NaNs in DataFrame
- redesign mapping 100x signals to ranges:
- no change for signals before last-active-trading-interval
- but for signals after last-active-trading-interval, process in reverse order
Improvements:
- increase max reconstruction depth from 1 to 2. E.g. now 1wk can be repaired with 1h (1wk->1d->1h)
Several improvements to price repair
Repair 100x and split errors:
- Handle stocks that recently suspended - use latest ACTIVE trading as baseline
- Improve error identification:
- Restrict repair to no older than 1 year before oldest split
- To reduce false positives when checking for multiday split errors,
only analyse 'Open' and 'Close' and use average change instead of nearest-to-1
- For weekly intervals reduce threshold to 3x standard deviation (5x was too high),
and for monthly increase to 6x
- For multiday intervas, if errors only detected in 1 column then assume false positive => ignore
Repair missing div-adjust:
- Fix repair of multiday intervals containing dividend
Price reconstruction:
- Move to after repairing 100x and split errors, so calibration works properly
- Fix maximum depth and reduce to 1
- Restrict calibration to 'Open' and 'Close', because 'Low' and 'High' can differ significantly between e.g. 1d and day-of-1h
Miscellaneous:
- Deprecate repair='silent', the logging module handles this
- Improve tests for 100x and split errors
- New test for 'repair missing div adjust'
Improve split-repair of multi-day intervals. Because split error can occur within a multi-day interval, e.g. mid-way through week, need to repair each OHLC column separately
Increase robustness of repair 'Adj Close'
Limit price-repair recursion depth to 2
