SolarPi – A Flask powered photovoltaic monitor

After collecting some photovoltaic data using PikoPy and a some readings from the residential meter it was time to put everything together. The data is collected by a couple of scripts triggered by a cronjob every five minutes.

$ crontab -l
*/5 * * * * python /home/solarpi/kostal_piko.py
*/5 * * * * python /home/solarpi/collect_meter.py
*/15 * * * * python /home/solarpi/collect_weather.py

The results are then written into a SQLite database. I chose SQLite over PostgreSQL for its lighter footprint on the Raspberry Pi. And since we will have no concurrent writes SQLite should be capable serving as a database backend.

The sole purpose of the web application is to fetch data from the database and render the results. I decided to use Flask, a microframework for Python based on Werkzeug and Jinja 2. For the Frontend I used the Bootstrap based SB Admin 2 Theme.

Initially I used SQLAlchemy as an Object Relational Mapper where I defined my models in Python:

class PVData(SurrogatePK, Model):
__tablename__ = 'pv_data'
id = Column(db.Integer(), nullable=False, primary_key=True)
created_at = Column(db.Text(), nullable=False, default=dt.datetime.utcnow)
dc_1_u = Column(db.Integer(), nullable=True)
dc_1_i = Column(db.Float(), nullable=True)
ac_1_u = Column(db.Integer(), nullable=True)
ac_1_p = Column(db.Integer(), nullable=True)
dc_2_u = Column(db.Integer(), nullable=True)
.
.
.
current_power = Column(db.Integer(), nullable=True)
daily_energy = Column(db.Float(), nullable=True)
total_energy = Column(db.Integer(), nullable=True)

While using an ORM is very helpful for CUD operation I decided to keep my read queries using raw SQL instead.

The main view is a dashboard presenting all kinds of interesting KPIs like yielded, imported, exported Energy and how much money was received for the sold energy:

SolarPi Dashboard

Furthermore there are detailed charts available for each day at a five minute interval, or the last 7 days and 30 days, respectively:

<a href="https://i2.wp.com/blog.tafkas.net/wp-content/uploads/SolarPiDailyChart.png”>SolarPi Daily Chart

The project is live at: http://solarpi.tafkas.net

The code is on Github at https://github.com/Tafkas/solarpi

  • BV123

    Would it be possible to use this to get the logs off a Morningstar MPPT controller? They use MODBUS to output the data.

    • This should be possible. Technically the fetching of the data is totally independent from the visualization. In my case I fetch the data from a Kostal Piko using my own library pikopy and store it in a SQLite database. You would need a script that that gathers the data from the MPPT controller. Once it is in the database it should be easy to connect the SolarPi application.

  • Joao Ferreira

    I have a kako 7900 and a raspberry pi.I would be the happiest man in the world if i could communicate between the pi and the kako. If i could put this info in a web server would be even better. Can you help or at least show me the way?…

  • Björn Rathjens

    I’m missing some kind of documentation how to get it up and running. i did not even find a howto setup the code, especially the sh-files aren’t findable for me. it would be great if you could give a short howto

  • Rüdiger Hopper Bentz

    Very interested in getting this to run with my SMA.
    In spite of reading up a lot about flask, gunicorn and the like I don’t manage to start the application.
    Can you give a clue with what parameters to start the application?
    Thanks in advance ?

    • Locally you can run it with “python manage.py server”

      • Rüdiger Hopper Bentz

        Thx for the reply. I figured that out by now 😉
        Got it running so far, the challenge for me is to remove the electricity meter since our older solar system sells all energy and we don’t have a smartmeter.
        Thanks to the well structured code it shouldn’t be too difficult.

  • Juergen Schorb

    Hi, a very nice and interessting application. i would use your application to visualizing my
    Solaredge inverter. Can you give me some hints, to get your application up and running in a apache web server?
    – apache config ( mod_python )
    – copy your files only in the document root?
    – …

    • If you just run it yourself on a local machine just use the development server with “python manage.py server”. If you want to run it in “production” I would use nginx and gunicorn.

  • Rampušák

    Dear Tafkas, I launched server from folder solarpi via “manage.py –host 0.0.0.0 runserver” but I not have created SQL tables. Exist any script for create tables used in this excellent project? Where exists the PY scripts: kostal_piko.py, collect_meter.py, collect_weather.py ? Many, many thanks for Your help.

  • Rampušák

    Dear Tafkas, I am tried to run this project on Windows machine with Python and other extras too. I am get same results that like on RPi. I do not have script for create DB in SQLite. The web server with run python returning error about missing table weather_data. Do you can help me with other information how I can made the tables for data ? Many, many thanks for your help.

    • I updated the migrations real quick:
      https://github.com/Tafkas/solarpi/commit/9355be7aeb6cc9c3787fff7acb7ba03a412df303

      Just run the “python manage.py db upgrade” and alembic will set-up your SQLite database.

      • Rampušák

        Great Man! But in subdirectory migrations missing file env.py. I must make this file with command “manage.py db init” ?? Many thanks for your help! 🙂

        • Rüdiger Hopper Bentz

          Did you get this to work?

          • Rampušák

            Yes, it working. But I have a problem with rendering charts in tab Charts and tab Yearly charts generate error 500.

      • Rampušák

        Next problem. In file settings.py the property SECRET_KEY refers to the envrionment variable SOLARPI_SECRET but I not have this envrionment variable and this key. I am find any string and used it.

        Server is runing but the web client show error:

        AttributeError
        AttributeError: ‘NoneType’ object has no attribute ‘current_power’

        Traceback (most recent call last):
        File “C:Python27libsite-packagesflaskapp.py”, line 1836, in __call__
        return self.wsgi_app(environ, start_response)
        File “C:Python27libsite-packagesflaskapp.py”, line 1820, in wsgi_app
        response = self.make_response(self.handle_exception(e))
        File “C:Python27libsite-packagesflaskapp.py”, line 1403, in handle_exception
        reraise(exc_type, exc_value, tb)
        File “C:Python27libsite-packagesflaskapp.py”, line 1817, in wsgi_app
        response = self.full_dispatch_request()
        File “C:Python27libsite-packagesflaskapp.py”, line 1477, in full_dispatch_request
        rv = self.handle_user_exception(e)
        File “C:Python27libsite-packagesflaskapp.py”, line 1381, in handle_user_exception
        reraise(exc_type, exc_value, tb)
        File “C:Python27libsite-packagesflaskapp.py”, line 1475, in full_dispatch_request
        rv = self.dispatch_request()
        File “C:Python27libsite-packagesflask_debugtoolbar__init__.py”, line 125, in dispatch_request
        return view_func(**req.view_args)
        File “C:Python27Projectssolarpisolarpipublicviews.py”, line 38, in home
        current_power = pv.current_power
        AttributeError: ‘NoneType’ object has no attribute ‘current_power’

        Tafkas, do you have any idea for?

        Many thanks for your help!

      • Rüdiger Hopper Bentz

        Thx ?

        • Rampušák

          A problem with AttributeError was that the tables was empty. I fill data with calling collect_kostal.py, collect_weather.py.
          But in publicviews.py I retype code and add test data with if… that any data I don’t in tables. For example electricity_data, last year data, etc.

          Now this project is working on Windows machine. I test it on RPi later.

          Great Man Tafkas, I wish you nice day!

      • Rampušák

        Problem with AttributeError was that the tables was empty. I fill data with calling collect_kostal.py, collect_weather.py.
        But in publicviews.py I retype code and add test data with if… that any data I don’t in tables. For example electricity_data, last year data, etc. Now this project is working on Windows machine. I test it on RPi later.
        Great Man Tafkas, I wish you nice day!

      • Rüdiger Hopper Bentz

        Thx a lot

      • Rampušák

        On pages with graphs is not be rendered. If I look to the datail of content web page in client then I see that in html page are values from data but client not render lines and columns of graphs. Do you have any idea why the graphs is not rendered?

      • Rampušák

        Dear Tafkas, do you have any idea why is not rendering the graphs on tab charts? And sub tab yearly cahrt generate web page with error 500. I am import data to table pvdata for last 3 years back.

  • Markus

    Super.
    Leider weiß ich nicht richtig wie man das installiert auf dem Raspberry.

    Mit einem Kostal PIKO Wechselrichter.
    Können sie mir eine Anleitung geben ? Wäre sehr nett.

  • Markus

    Python 2.7.9 (default, Mar 8 2015, 00:52:26)
    [GCC 4.9.2] on linux2
    Type “copyright”, “credits” or “license()” for more information.
    >>> ================================ RESTART ================================
    >>>

    Traceback (most recent call last):
    File “/home/pi/Downloads/collect_kostal.py”, line 93, in
    main()
    File “/home/pi/Downloads/collect_kostal.py”, line 87, in main
    data = get_data()
    File “/home/pi/Downloads/collect_kostal.py”, line 54, in get_data
    dc_i_3 = data[13]
    IndexError: list index out of range
    >>>

    • Rampušák

      The web page of your inverter is different that pages for which was made the collect script.
      Probably your inverter is for 1 phase. man Tafkas and I too we have the inverters for 3 electrical phases.

      You must modify the script.

      • Markus

        Can you help me by the modify ?

        • Rampušák

          You upload the printscreen of web page of your inverter.

          • Markus

            ok

          • Rampušák

            You have configuration for 2 strings. We have configuration for 3 strings. You modify the script collect_kostal.py from line number 52:

            dc_u_3 = 0
            ac_u_3 = data[11]
            dc_i_3 = 0
            ac_p_3 = data[12]

          • Markus

            That not work.
            no such table pvdata

          • Rampušák

            Probably you not created the database app.db or dev.db in your installation. Look in another message in this discussion. You must first run the command: “manage.py db upgrade”

          • Markus

            pi@raspberrypi:~/Downloads/solarpi-master $ python manage.py db upgrade
            Traceback (most recent call last):
            File “manage.py”, line 9, in
            from solarpi.app import create_app
            File “/home/pi/Downloads/solarpi-master/solarpi/app.py”, line 5, in
            from solarpi.settings import ProdConfig
            File “/home/pi/Downloads/solarpi-master/solarpi/settings.py”, line 7, in
            class Config(object):
            File “/home/pi/Downloads/solarpi-master/solarpi/settings.py”, line 8, in Config
            SECRET_KEY = os_env[‘SOLARPI_SECRET’]
            File “/usr/lib/python2.7/UserDict.py”, line 23, in __getitem__
            raise KeyError(key)
            KeyError: ‘SOLARPI_SECRET’

          • Markus

            http://flask.pocoo.org/docs/0.10/quickstart/#sessions stand in a other forum as tipp.
            But i not understand what is to do.

  • Markus

    Can anyone upload a raspberry pi image with a working installed SolarPi Setup ?

  • Markus

    python collect_kostal.py
    Traceback (most recent call last):
    File “collect_kostal.py”, line 93, in
    main()
    File “collect_kostal.py”, line 89, in main
    save_data_to_db(data)
    File “collect_kostal.py”, line 71, in save_data_to_db
    cur.execute(query, tuple([datetime.datetime.now().isoformat()] + data))
    sqlite3.OperationalError: no such table: pvdata

    • Did you run the database migration? “python manage.py db upgrade”

      • Markus

        I have try it but i not understand.
        Can you say how that works ?
        (Habe ich versucht aber weiß nicht genau welche Befehle ich dafür benutzen muss)
        (Es wäre einfach sehr gut wenn sie man eine genauere Programm Beschreibung und Installations Anleitung schreiben würden)

  • John

    Hi tafkas! Well done for this work! I want to try this project but for a big pv plant with many inverters (more than twenty). Do you think RPI can do this ?
    Thanks a lot!
    John

  • Saamir

    I got solarpi working on my Raspberry Pi, and wrote a howto to document the steps. It might be good to include a howto in the repository, and perhaps the source code changes I had to make. The howto is at:
    http://infrontpower.com/solarpi/howto-en.txt
    If you have any problems with it, or suggestions to improve, reply here.

    • Adam

      I am getting the following when i try your steps.

      pi@raspberrypi:~/solarpi $ SOLARPI_ENV=dev SOLARPI_SECRET=”xyz” python manage.py db upgrade
      Traceback (most recent call last):
      File “manage.py”, line 9, in
      from solarpi.app import create_app
      File “/home/pi/solarpi/solarpi/app.py”, line 10, in
      from solarpi.settings import ProdConfig
      File “/home/pi/solarpi/solarpi/settings.py”, line 7, in
      class Config(object):
      File “/home/pi/solarpi/solarpi/settings.py”, line 17, in Config
      SENTRY_DNS = os_env[‘SENTRY_DNS’]
      File “/usr/lib/python2.7/UserDict.py”, line 23, in __getitem__
      raise KeyError(key)
      KeyError: ‘SENTRY_DNS’

      • Saamir

        I believe the error with line 148 is just a warning, and shouldn’t cause the patch to fail.
        Line 47 is a failure, and the only reason I can think of is that you don’t have the same Solarpi version that I used. Try the “git checkout 23aa8c4” as mentioned in the howto.
        The KeError traceback is because the settings.py file was not patched. That may be due to the previous problem.

        Please let me know how this turns out, so I can modify the howto for others. Thanks.

        • Adam

          I deleted it all and started again and get the same problem.

          pi@raspberrypi:~ $ rm -rf solarpi.patch
          pi@raspberrypi:~ $ sudo vi solarpi.patch
          pi@raspberrypi:~ $ pwd
          /home/pi
          pi@raspberrypi:~ $ git clone https://github.com/Tafkas/solarpi.git
          Cloning into ‘solarpi’…
          remote: Counting objects: 1909, done.
          remote: Total 1909 (delta 0), reused 0 (delta 0), pack-reused 1909
          Receiving objects: 100% (1909/1909), 1.33 MiB | 188.00 KiB/s, done.
          Resolving deltas: 100% (1154/1154), done.
          Checking connectivity… done.
          pi@raspberrypi:~ $ cd solarpi/
          pi@raspberrypi:~/solarpi $ git checkout 23aa8c4
          Note: checking out ’23aa8c4’.

          You are in ‘detached HEAD’ state. You can look around, make experimental
          changes and commit them, and you can discard any commits you make in this
          state without impacting any branches by performing another checkout.

          If you want to create a new branch to retain commits you create, you may
          do so (now or later) by using -b with the checkout command again. Example:

          git checkout -b new_branch_name

          HEAD is now at 23aa8c4… add .gitattributes
          pi@raspberrypi:~/solarpi $ git apply ~/solarpi.patch
          /home/pi/solarpi.patch:148: trailing whitespace.

          error: patch failed: solarpi/charts/views.py:47
          error: solarpi/charts/views.py: patch does not apply
          /home/pi/solarpi.patch:148: new blank line at EOF.
          +
          pi@raspberrypi:~/solarpi $ cp /usr/local/lib/python2.7/dist-packages/alembic/templates/generic/env.py migrations/
          pi@raspberrypi:~/solarpi $
          pi@raspberrypi:~/solarpi $ SOLARPI_ENV=dev SOLARPI_SECRET=”xyz” python manage.py db upgrade
          Traceback (most recent call last):
          File “manage.py”, line 9, in
          from solarpi.app import create_app
          File “/home/pi/solarpi/solarpi/app.py”, line 10, in
          from solarpi.settings import ProdConfig
          File “/home/pi/solarpi/solarpi/settings.py”, line 7, in
          class Config(object):
          File “/home/pi/solarpi/solarpi/settings.py”, line 17, in Config
          SENTRY_DNS = os_env[‘SENTRY_DNS’]
          File “/usr/lib/python2.7/UserDict.py”, line 23, in __getitem__
          raise KeyError(key)
          KeyError: ‘SENTRY_DNS’
          pi@raspberrypi:~/solarpi $

          • Saamir

            OK, I figured it out. The problem is cutting and pasting from the web page. I think there’s a unicode character in it that won’t copy. So I updated the howto on the web page. Anybody who tries it, please let me know how it worked so I can continue to update the howto. Next time I’ll try to respond a little more timely. :/