misc improvement to docs

README.md

@@ -36,7 +36,7 @@ some novel experimental features like exponentially-weighted covariance matrices
 
 It is **extensive** yet easily **extensible**, and can be useful for both the casual investor and the serious practitioner. Whether you are a fundamentals-oriented investor who has identified a
 handful of undervalued picks, or an algorithmic trader who has a basket of
-interesting signals, PyPortfolioOpt can help you combine your alpha streams
+strategies, PyPortfolioOpt can help you combine your alpha sources
 in a risk-efficient way.
 
 Head over to the [documentation on ReadTheDocs](https://pyportfolioopt.readthedocs.io/en/latest/) to get an in-depth look at the project, or check out the [cookbook](https://github.com/robertmartin8/PyPortfolioOpt/tree/master/cookbook) to see some examples showing the full process from downloading data to building a portfolio.
@@ -62,7 +62,6 @@ Head over to the [documentation on ReadTheDocs](https://pyportfolioopt.readthedo
   - [Other optimisers](#other-optimisers)
 - [Advantages over existing implementations](#advantages-over-existing-implementations)
 - [Project principles and design decisions](#project-principles-and-design-decisions)
-- [Roadmap](#roadmap)
 - [Testing](#testing)
 - [Contributing](#contributing)
 - [Getting in touch](#getting-in-touch)
@@ -231,7 +230,7 @@ components while still making use of the framework that PyPortfolioOpt provides.
 
 ## Features
 
-In this section, we detail PyPortfolioOpt's current available functionality as per the above breakdown. More examples are offered in the Jupyter notebooks [here](https://github.com/robertmartin8/PyPortfolioOpt/tree/master/cookbook). Another good resource is the [tests](https://github.com/robertmartin8/PyPortfolioOpt/tree/master/tests).
+In this section, we detail some of PyPortfolioOpt's available functionality. More examples are offered in the Jupyter notebooks [here](https://github.com/robertmartin8/PyPortfolioOpt/tree/master/cookbook). Another good resource is the [tests](https://github.com/robertmartin8/PyPortfolioOpt/tree/master/tests).
 
 A far more comprehensive version of this can be found on [ReadTheDocs](https://pyportfolioopt.readthedocs.io/en/latest/), as well as possible extensions for more advanced users.
 
@@ -363,19 +362,6 @@ Please refer to the [documentation](https://pyportfolioopt.readthedocs.io/en/lat
 - Formatting should never get in the way of coding: because of this,
   I have deferred **all** formatting decisions to [Black](https://github.com/ambv/black).
 
-## Roadmap
-
-Feel free to raise an issue requesting any new features – here are some of the things I want to implement:
-
-- Optimising for higher moments (i.e skew and kurtosis)
-- Factor modelling: doable but not sure if it fits within the API.
-- Proper CVaR optimisation – remove NoisyOpt and use linear programming
-- More objective functions, including the Calmar Ratio, Sortino Ratio, etc.
-- Monte Carlo optimisation with custom distributions
-- Open-source backtests using either `Backtrader <https://www.backtrader.com/>`_ or
-  `Zipline <https://github.com/quantopian/zipline>`_.
-- Further support for different risk/return models
-
 ## Testing
 
 Tests are written in pytest (much more intuitive than `unittest` and the variants in my opinion), and I have tried to ensure close to 100% coverage. Run the tests by navigating to the package directory and simply running `pytest` on the command line.
@@ -401,6 +387,20 @@ been tested to work as intended.
 
 Contributions are *most welcome*. Have a look at the [Contribution Guide](https://github.com/robertmartin8/PyPortfolioOpt/blob/master/CONTRIBUTING.md) for more.
 
+I'd like to thank all of the people who have contributed to PyPortfolioOpt since its release in 2018.
+Special shout-outs to:
+
+- Philipp Schiele
+- Carl Peasnell
+- Felipe Schneider
+- Dingyuan Wang
+- Pat Newell
+- Aditya Bhutra
+- Thomas Schmelzer
+- Rich Caputo
+
 ## Getting in touch
 
-If you would like to reach out for any reason, be it consulting opportunities or just for a chat, please do so via the [form](https://reasonabledeviations.com/about/) on my website.
+If you are having a problem with PyPortfolioOpt, please raise an issue.
+
+For anything else, you can contact me via the [form](https://reasonabledeviations.com/about/) on my website.

docs/BlackLitterman.rst

@@ -225,6 +225,7 @@ Documentation reference
 
 .. automodule:: pypfopt.black_litterman
     :members:
+    :exclude-members: BlackLittermanModel
 
     .. autoclass:: BlackLittermanModel
         :members:

docs/EfficientFrontier.rst

@@ -128,7 +128,8 @@ Basic Usage
 
     PyPortfolioOpt defers to cvxpy's default choice of solver. If you would like to explicitly
     choose the solver, simply pass the optional ``solver = "ECOS"`` kwarg to the constructor.
-    You can choose from any of the `supported solvers <https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver>`_.
+    You can choose from any of the `supported solvers <https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver>`_,
+    and pass in solver params via ``solver_options`` (a ``dict``). 
 
 Adding objectives and constraints
 =================================
@@ -137,7 +138,7 @@ EfficientFrontier inherits from the BaseConvexOptimizer class. In particular, th
 add constraints and objectives are documented below:
 
 
-    .. class:: pypfopt.base_optimizer.BaseConvexOptimizer
+.. class:: pypfopt.base_optimizer.BaseConvexOptimizer
 
     .. automethod:: add_constraint
 
@@ -153,8 +154,6 @@ Objective functions
     :members:
 
 
-One of the experimental features implemented in PyPortfolioOpt is the L2 regularisation
-parameter ``gamma``, which is discussed below.
 
 .. _L2-Regularisation:
 
@@ -194,21 +193,80 @@ used to make them larger).
     increase ``gamma``.
 
 
+Efficient Semivariance
+======================
+
+The mean-variance optimisation methods described above can be used whenever you have a vector
+of expected returns and a covariance matrix. Most of the functions provided in :ref:`risk-models`
+are really just different ways of estimating the covariance matrix.
+
+However, you may want to construct the efficient frontier for an entirely different risk model.
+For example, instead of penalising volatility (which is symmetric), you may want to penalise
+only the downside deviation. Stocks that frequently jump upwards might be desirable for you!
+
+As of v1.3.0, PyPortfolioOpt offers this functionality via the :py:class:`EfficientSemivariance` class. 
+:py:class:`EfficientSemivariance` inherits from :py:class:`EfficientFrontier`, so it has the same utility methods
+(e.g :py:func:`add_constraint`, :py:func:`portfolio_performance`), but finds portfolios on the mean-semivariance
+frontier. Note  that some of the parent methods, like :py:func:`max_sharpe` and :py:func:`min_volatility`
+are not applicable to mean-semivariance portfolios, so calling them returns an error.
+
+:py:class:`EfficientSemivariance` has a slightly different API to :py:class:`EfficientFrontier`. Instead of passing
+in a covariance matrix, you should past in a dataframe of historical returns (this can be constructed
+from your price dataframe using the helper method :py:func:`expected_returns.returns_from_prices`). Here
+is a full example, in which we seek the portfolio that minimises the semivariance for a target
+annual return of 20%::
+
+    from pypfopt import expected_returns, EfficientSemivariance
+
+    df = ... # your dataframe of prices
+    mu = expected_returns.mean_historical_returns(df)
+    historical_returns = expected_returns.returns_from_prices(df)
+
+    es = EfficientSemivariance(mu, historical_returns)
+    es.efficient_return(0.20)
+
+    # We can use the same helper methods as before
+    weights = es.clean_weights() 
+    print(weights)
+    es.portfolio_performance(verbose=True)
+
+The ``portfolio_performance`` method outputs the expected portfolio return, semivariance,
+and the Sortino ratio (like the Sharpe ratio, but for downside deviation).
+
+Interested readers should refer to Estrada (2007) [2]_ for more details. I'd like to thank 
+`Philipp Schiele <https://github.com/phschiele>`_ for authoring the bulk
+of the efficient semivariance functionality (all errors are my own).
+
+.. caution::
+
+    Finding portfolios on the mean-semivariance frontier is computationally harder
+    than standard mean-variance optimisation. While :py:class:`EfficientSemivariance` allows for
+    additional constraints/objectives in principle, you are much more likely to run into
+    solver errors. I suggest that you keep :py:class:`EfficientSemivariance` problems small
+    and minimally constrained.
+
+.. autoclass:: pypfopt.efficient_frontier.EfficientSemivariance
+    :members:
+    :exclude-members: max_sharpe, min_volatility
+
+
+
 .. _custom-optimisation:
 
 Custom optimisation problems
 ============================
 
 Previously we described an API for adding constraints and objectives to one of the core
-optimisation problems in the ``EfficientFrontier`` class. However, what if you aren't interested
+optimisation problems in the :py:class:`EfficientFrontier` class. However, what if you aren't interested
 in anything related to ``max_sharpe()``, ``min_volatility()``, ``efficient_risk()`` etc and want to
 set up a completely new problem to optimise for some custom objective?
 
-The ``EfficientFrontier`` class inherits from the ``BaseConvexOptimizer``, which allows you to
+The :py:class:`EfficientFrontier` class inherits from the ``BaseConvexOptimizer``, which allows you to
 define your own optimisation problem. You can either optimise some generic ``convex_objective``
 (which *must* be built using ``cvxpy`` atomic functions -- see `here <https://www.cvxpy.org/tutorial/functions/index.html>`_)
 or a ``nonconvex_objective``, which uses ``scipy.optimize`` as the backend and thus has a completely
-different API. For examples, check out this `cookbook recipe <https://github.com/robertmartin8/PyPortfolioOpt/blob/master/cookbook/3-Advanced-Mean-Variance-Optimisation.ipynb>`_
+different API. For examples, check out this `cookbook recipe
+<https://github.com/robertmartin8/PyPortfolioOpt/blob/master/cookbook/3-Advanced-Mean-Variance-Optimisation.ipynb>`_.
 
     .. class:: pypfopt.base_optimizer.BaseConvexOptimizer
 
@@ -221,4 +279,4 @@ References
 ==========
 
 .. [1] Boyd, S.; Vandenberghe, L. (2004). `Convex Optimization <https://web.stanford.edu/~boyd/cvxbook/>`_.
-
+.. [2] Estrada, J (2007). `Mean-Semivariance Optimization: A Heuristic Approach <https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1028206>`_.

docs/Roadmap.rst

@@ -8,18 +8,28 @@ Roadmap and Changelog
 Roadmap
 =======
 
-These are some of the things that I am thinking of adding in the near future. If you
-have any other feature requests, please raise them using GitHub
+These are some of the features that I think would greatly improve PyPortfolioOpt; if you
+are interested in implementing one of these, raise an issue or send me an email and we can
+discuss. If you have any other feature requests, please raise them using GitHub
 `issues <https://github.com/robertmartin8/PyPortfolioOpt/issues>`_
 
-- Optimising for higher moments (i.e skew and kurtosis)
-- Factor modelling: doable but not sure if it fits within the API.
-- Proper CVaR optimisation – remove NoisyOpt and use linear programming
-- Monte Carlo optimisation with custom distributions
+- Improved plotting
 - Open-source backtests using either `Backtrader <https://www.backtrader.com/>`_ or
   `Zipline <https://github.com/quantopian/zipline>`_.
+- Optimising for higher moments (i.e skew and kurtosis)
+- Factor modelling - this is conceptually doable, but a lot of thought needs to be put into the API.
+- CVaR optimisation
+- Monte Carlo optimisation with custom distributions
 - Further support for different risk/return models
 
+1.3.0
+=====
+
+- Efficient semivariance portfolios (thanks to `Philipp Schiele <https://github.com/phschiele>`_)
+- Improved functionality for portfolios with short positions (thanks to `Rich Caputo <https://github.com/arcaputo3>`_).
+- Significant improvement in test coverage (thanks to `Carl Peasnell <https://github.com/SeaPea1>`_).
+- Several bug fixes and usability improvements.
+
 1.2.0
 =====
 
@@ -30,7 +40,6 @@ have any other feature requests, please raise them using GitHub
 - Adding new cookbook for examples (in progress).
 - Packaging: added bettter instructions for windows, added docker support.
 
-
 1.2.1
 -----
 
@@ -57,8 +66,8 @@ Matplotlib now required dependency; support for pandas 1.0.
 1.2.5
 -----
 
-- Fixed compounding in ``expected_returns`` (thanks to Aditya Bhutra).
-- Improvements in advanced cvxpy API (thanks to Pat Newell).
+- Fixed compounding in ``expected_returns`` (thanks to `Aditya Bhutra <https://github.com/bhutraaditya>`_).
+- Improvements in advanced cvxpy API (thanks to `Pat Newell <https://github.com/pmn4>`_).
 - Deprecating James-Stein
 - Exposed ``linkage_method`` in HRP. 
 - Added support for cvxpy 1.1.

docs/UserGuide.rst

@@ -5,7 +5,7 @@ User Guide
 ##########
 
 This is designed to be a practical guide, mostly aimed at users who are interested in a
-quick way of optimally combining some assets (most likely equities). However, when
+quick way of optimally combining some assets (most likely stocks). However, when
 necessary I do introduce the required theory and also point out areas that may be
 suitable springboards for more advanced optimisation techniques. Details about the
 parameters can be found in the respective documentation pages (please see the sidebar).
@@ -51,7 +51,7 @@ of the GitHub repo.
 
 .. note::
 
-    Pricing data does not have to be daily, but the frequency should ideally
+    Pricing data does not have to be daily, but the frequency should
     be the same across all assets (workarounds exist but are not pretty).
 
 After reading your historical prices into a pandas dataframe ``df``, you need to decide
@@ -88,7 +88,7 @@ Efficient Frontier Optimisation
 ===============================
 
 Efficient Frontier Optimisation is based on Harry Markowitz's 1952 classic paper [1]_, which
-turned portfolio management from an art into a science. The key insight is that by
+spearheaded the transformation of portfolio management from an art into a science. The key insight is that by
 combining assets with different expected returns and volatilities, one can decide on a
 mathematically optimal allocation.
 
@@ -287,10 +287,10 @@ should you try?
 - Try the Hierarchical Risk Parity model (see :ref:`other-optimisers`) – which seems
   to robustly outperform mean-variance optimisation out of sample.
 - Use the Black-Litterman model to construct a more stable model of expected returns.
-  Alternatively, just drop the expected returns altogether!. There is a large body of research
+  Alternatively, just drop the expected returns altogether! There is a large body of research
   that suggests that minimum variance portfolios (``ef.min_volatility()``) consistently outperform
-  maximum Sharpe ratio portfolios out-of-sample, because of the difficulty of forecasting expected returns.
-- Try different risk models: different asset classes may require different risk models.
+  maximum Sharpe ratio portfolios out-of-sample (even when measured by Sharpe ratio), because of the difficulty of forecasting expected returns.
+- Try different risk models: shrinkage models are known to have better numerical properties compared with the sample covariance matrix.
 - Add some new objective terms or constraints. Tune the L2 regularisation parameter to see how diversification
   affects the performance.
 

docs/index.rst

@@ -36,7 +36,7 @@ It is **extensive** yet easily
 **extensible**, and can be useful for both the casual investor and the serious
 practitioner. Whether you are a fundamentals-oriented investor who has identified a
 handful of undervalued picks, or an algorithmic trader who has a basket of
-interesting signals, PyPortfolioOpt can help you combine your alpha-generators
+strategies, PyPortfolioOpt can help you combine your alpha sources
 in a risk-efficient way.
 
 
@@ -48,8 +48,8 @@ Installation on macOS or linux is as simple as::
     pip install PyPortfolioOpt
 
 Windows users need to go through the additional step of downloading C++ (for ``cvxpy``). You can
-download this `here <https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=16>`_,
-with additional instructions `here <https://drive.google.com/file/d/0B4GsMXCRaSSIOWpYQkstajlYZ0tPVkNQSElmTWh1dXFaYkJr/view>`_.
+download this `here <https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=16>`__,
+with additional instructions `here <https://drive.google.com/file/d/0B4GsMXCRaSSIOWpYQkstajlYZ0tPVkNQSElmTWh1dXFaYkJr/view>`__.
 
 For the sake of best practice, it is good to do this with a dependency manager. I suggest you
 set yourself up with `poetry <https://github.com/sdispater/poetry>`_, then within a new poetry project
@@ -193,6 +193,19 @@ Project principles and design decisions
   <https://github.com/ambv/black>`_.
 
 
+Contributors
+=============
+
+This is a non-exhaustive unordered list of contributors:
+
+- Philipp Schiele
+- Carl Peasnell
+- Felipe Schneider
+- Dingyuan Wang
+- Pat Newell
+- Aditya Bhutra
+- Thomas Schmelzer
+- Rich Caputo
 
 
 Indices and tables

pypfopt/base_optimizer.py

@@ -129,6 +129,7 @@ class BaseConvexOptimizer(BaseOptimizer):
     - ``tickers`` - str list
     - ``weights`` - np.ndarray
     - ``solver`` - str
+    - ``solver_options`` - {str: str} dict
 
     Public methods:
 

pypfopt/discrete_allocation.py

@@ -56,8 +56,9 @@ class DiscreteAllocation:
         :type latest_prices: pd.Series
         :param total_portfolio_value: the desired total value of the portfolio, defaults to 10000
         :type total_portfolio_value: int/float, optional
-        :param short_ratio: the short ratio, e.g 0.3 corresponds to 130/30
-        :type short_ratio: float
+        :param short_ratio: the short ratio, e.g 0.3 corresponds to 130/30. If None,
+                            defaults to the input weights.
+        :type short_ratio: float, defaults to None.
         :raises TypeError: if ``weights`` is not a dict
         :raises TypeError: if ``latest_prices`` isn't a series
         :raises ValueError: if ``short_ratio < 0``
@@ -131,9 +132,9 @@ class DiscreteAllocation:
         using a greedy iterative approach.
 
         :param reinvest: whether or not to reinvest cash gained from shorting
-        :type reinvest: bool
+        :type reinvest: bool, defaults to False
         :param verbose: print error analysis?
-        :type verbose: bool
+        :type verbose: bool, defaults to False
         :return: the number of shares of each ticker that should be purchased,
                  along with the amount of funds leftover.
         :rtype: (dict, float)
@@ -251,14 +252,13 @@ class DiscreteAllocation:
             self._allocation_rmse_error(verbose)
         return self.allocation, available_funds
 
-
     def lp_portfolio(self, reinvest=False, verbose=False, solver="GLPK_MI"):
         """
         Convert continuous weights into a discrete portfolio allocation
         using integer programming.
 
         :param reinvest: whether or not to reinvest cash gained from shorting
-        :type reinvest: bool
+        :type reinvest: bool, defaults to False
         :param verbose: print error analysis?
         :type verbose: bool
         :param solver: the CVXPY solver to use (must support mixed-integer programs)

tests/utilities_for_tests.py

@@ -66,7 +66,6 @@ def setup_efficient_frontier(
 
 def setup_efficient_semivariance(data_only=False, solver=None, verbose=False):
     df = get_data().dropna(axis=0, how="any")
-    # TODO: figure out frequency
     mean_return = expected_returns.mean_historical_return(df, compounding=False)
     historic_returns = returns_from_prices(df)
     if data_only: