導入 Django 以架設物聯網中的資料紀錄網站

2020-01-18: 導入 Django

以架設物聯網中的資料紀錄網站

講者: 何岳峰 Amon Ho

hoamon@ho600.com

資源

簡報: https://www.ho600.com/presentations/20200118at768/contents.html

_images/frame.png

文件: https://book-for-import-django.readthedocs.io/zh_TW/latest/01part.html 程式碼: https://github.com/ho600-ltd/examples-of-book-for-import-django

欲解決問題: MQTT Subscriber 函式所收到的資料要如何處理?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import logging
import paho.mqtt.client as mqtt

def post_data(*args, **kw):
    """
        How to program this function?
    """
    pass

def on_connect(client, userdata, flags, rc):
    lg = logging.getLogger('info')
    lg.debug("Connected with result code: {}".format(rc))
    client.subscribe("ho600/office/power1")

def on_message(client, userdata, msg):
    lg = logging.getLogger('info')
    lg.debug("{} {}".format(msg.topic, msg.payload))
    pos_data(msg)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("my-iot.domain.com", 1883, 60)
client.loop_forever()

問題分析: post_data 該把資料寫到那裡?

  1. 寫進本地端檔案:
    • 寫入權限
    • 格式
  2. 寫進某個資料庫(SQLite, MariaDB, PostgreSQL, SQL Server, …):
    • 要有 host, username, password, database name, table name 及 table schema
    • 對資料表的操作權限
  3. 寫進遠端 http(s) 網站:
    • path, querystring, request body, content_type
    • api key, 權限
  4. 應該使用 RESTful API 網站,資料表的 CRUD 操作就是對應 HTTP POST, GET, PATCH/PUT, DELETE Methods

初始化開發環境:使用工具/函式庫/資料庫管理系統/...

  • 程式編輯器: Visual Studio Code
  • 版本控制器: git
  • 套件管理工具: scoop(Windows PowerShell)/brew(macos)/apt(ubuntu)
  • 資料庫管理系統: MariaDB
  • Python3
  • Django-2.2.x
  • virtualenv
  • django-guardian
  • django-restframework
_images/vsc.png
  • 方便、高級的 Vim 編輯器
  • 檔案管理樹
  • Shell console

版本控制器絕對是程式設計師必備技能

目前常見的有 git, mercurial, svn ,而 git 是目前最熱門的

本範例程式碼放在 https://github.com/ho600-ltd/examples-of-book-for-import-django/

_images/github.png

套件管理工具

scoop for Windows PowerShell: https://scoop.sh/

1
2
3
4
5
6
7
8
9
PS> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
PS> Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
PS> scoop install aria2
PS> scoop bucket add extras
PS> Install-Module -Name PowerShellGet -Force Exit # with administrator permission
PS> PowerShellGet\Install-Module posh-git -Scope CurrentUser -AllowPrerelease -Force # with general user
PS> Add-PoshGitToProfile -AllUsers -AllHosts # with administrator permission
PS> # just add a line break
PS> scoop install virtualenv

brew for macos: https://brew.sh/

1
2
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew install virtualenv

apt for Ubuntu: 系統內建

1
2
$ sudo apt install python3-pip
$ pip3 install virtualenv

Databases

  • MySQL(中文問題) + PostgreSQL(中文沒問題)
  • MySQL
  • PostgreSQL
  • MariaDB 轉換中

Use Python3

Python2 已於 2020 年 1 月 1 日正式被棄用

print('Hello World!')

virtualenv

  • 隔離不同版本的 Python
  • 隔離不同版本的函式庫
  • 在同一台機器上運作多個網站
$ virtualenv -p python2 --no-site-packages py2env
$ source py2env/bin/activate
(py2env) $ pip install "Django>=1,<2"

$ virtualenv -p python3.7 --no-site-packages py37env
$ source py37env/bin/activate
(py37env) $ pip install "Django>=2,<3"

$ virtualenv -p python3.5 --no-site-packages py35env
$ source py35env/bin/activate
(py35env) $ pip install "Django>=2,<3"

django-restframework

建立一個 RESTful API Service

django-guardian

  • 權限可以細到「某人」在「某一筆紀錄」上可「R, U, C」
  • Django 內建的權限系統只能管到「某人」在「某 Model 」上的 CRUD

初始化 django-based 專案

1
2
3
4
5
6
7
8
9
$ virtualenv -p python3 --no-site-packages restful_api_site.py3env
$ source restful_api_site.py3env/bin/activate \
# In PowerShell: \
# PS C:\> restful_api_site.py3env\scripts\activate

(restful_api_site.py3env) $ pip install "Django>2.2,<2.3"
...
Successfully installed Django-2.2.9 sqlparse-0.3.0
(restful_api_site.py3env) $ django-admin startproject restful_api_site

git 操作

  • git init # 一次性
  • git add
  • git commit # shortcut: git ci
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
(restful_api_site.py3env) $ cd restful_api_site
(restful_api_site.py3env) restful_api_site/ $ ls
manage.py        restful_api_site
(restful_api_site.py3env) restful_api_site/ $ git init && git add . && git ci -m '...'
...
[master cc69bfb] ...
5 files changed, 126 insertions(+)
create mode 100644 ...
(restful_api_site.py3env) restful_api_site/ $ git di cc69bfb^..cc69bfb --name-only
restful_api_site/manage.py
restful_api_site/restful_api_site/__init__.py
restful_api_site/restful_api_site/settings.py
restful_api_site/restful_api_site/urls.py
restful_api_site/restful_api_site/wsgi.py

restful_api_site 專案從無到初始化的程式碼差異比對: https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/cc69bfb9

程式檔說明

  • manage.py: 在本地端開發時,用以執行一個 http deamon 的執行檔。 $ ./manage.py runserver
  • __init__.py: 為一空內容的純文字檔,置於第二層的 restful_api_site/ 中,這樣第二層的 resuful_api_site 可視為一個 module
  • settings.py: 專案的基本設定檔
  • urls.py: 當 restful_api_site 運作在 http deamon 或 WSGI deamon 上, urls.py 可載明進入的 url path 為何? 並對應到那些 view function
  • wsgi.py: 給 WSGI server 的進入點,讓 restful_api_site 運作在 WSGI server 上

settings.py

(restful_api_site.py3env) restful_api_site/ $ git di cc69bfb^..cc69bfb restful_api_site/settings.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
diff --git a/restful_api_site/restful_api_site/settings.py b/restful_api_site/restful_api_site/settings.py
new file mode 100644
index 0000000..5a8707d
--- /dev/null
+++ b/restful_api_site/restful_api_site/settings.py
@@ -0,0 +1,120 @@
+"""
+Django settings for restful_api_site project.
+Generated by 'django-admin startproject' using Django 2.2.9.
...
+"""
...
+ROOT_URLCONF = 'restful_api_site.urls'
...
+WSGI_APPLICATION = 'restful_api_site.wsgi.application'
+DATABASES = {    'default': {
+        'ENGINE': 'django.db.backends.sqlite3',
+        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),    }}
...
+LANGUAGE_CODE = 'en-us'
+TIME_ZONE = 'UTC'
+USE_I18N = True
+USE_L10N = True
+USE_TZ = True
+STATIC_URL = '/static/'

預設的網站程式碼可直接運作

此修改版本的 settings.py 內容可到 https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/cc69bfb98c16ffcb72d36dd15f23d0ccb72d9cf4#diff-21a51302934f25442a0bd16766f498df

在目前這個階段, restful_api_site 是一個擁有 django 預設功能的網站,而資料庫管理系統上預設是用 sqlite3 ,其設定方式在 settings.py :

1
2
3
4
5
6
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

運作網站的第一步是要建立資料庫結構

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
...
Applying contenttypes.0002_remove_content_type_name... OK
...
Applying sessions.0001_initial... OK

運作本地端 http deamon

1
2
3
4
5
6
(restful_api_site.py3env) restful_api_site/ $ ./manage.py runserver
...
January 17, 2020 - 03:35:27
Django version 2.2.9, using settings 'restful_api_site.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

因為 settings.LANGUAGE_CODE = “en-us” ,所以網頁是英文的

_images/default_index.png

修改 settings.py

  • LANGUAGE_CODE : 語言預設是使用正體中文
  • DATABASES[‘default’] : 資料庫則是改用 MariaDB
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
DATABASES = {
    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
+        'ENGINE': 'django.db.backends.mysql',
+        'NAME': 'restful_api_site',
+        'USER': 'restful_api_site',
+        'PASSWORD': 'restful_api_site_pw', # os.env['RESTFUL_API_SITE_PW']
+        'HOST': 'my.mariadb.host',
+        'PORT': '3306',
+        'OPTIONS': {
+        },
    }
}
...
-LANGUAGE_CODE = 'en-us'
+LANGUAGE_CODE = 'zh-Hant'

https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/ca533439e

設定 MariaDB

資料庫名, 使用者帳號, 密碼, 權限

1
2
3
4
5
6
$ mysql -h my.mariadb.host -u root -p
MariaDB [(none)]> CREATE DATABASE restful_api_site
    CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
MariaDB [(none)]> create user 'restful_api_site'@'%' identified by 'restful_api_site_pw';
MariaDB [(none)]> GRANT ALL PRIVILEGES on restful_api_site.* to restful_api_site@'%';
MariaDB [(none)]> \q

Install mysqlclient(MariaDB driver)

先紀錄到 restful_api_site/requirements.txt ,再安裝

1
2
3
# requirements.txt
Django>=2.2,<2.3
mysqlclient==1.4.5
1
2
3
4
5
6
(restful_api_site.py3env) restful_api_site/ $ pip install -r requirements.txt
...
Successfully installed mysqlclient-1.4.5
(restful_api_site.py3env) restful_api_site/ $ \
ls ../restful_api_site.py3env/lib/python3.7/site-packages/mysqlclient-1.4.5.dist-info
INSTALLER     LICENSE       METADATA      RECORD        WHEEL         top_level.txt

資料表生成

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
...
Applying contenttypes.0002_remove_content_type_name... OK
...
Applying sessions.0001_initial... OK

再運作本地端 http deamon

_images/zh_hant_index.png

到本階段為止,範例程式碼的進度在 https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/76c5dd8180e115124c26e0c911dfae27670749b1

Django Admin 操作

預設的 urls.py 有列入 Django Admin 模組的進入網址

1
2
3
4
5
from django.contrib import admin
from django.urls import path
urlpatterns = [
    path('admin/', admin.site.urls),
]

http://127.0.0.1:8000/admin/ ,可以看見一個登入頁:

_images/admin_login.png

透過 management command 來創建一個超級管理員帳戶

1
2
3
4
5
6
7
(restful_api_site.py3env) restful_api_site/ $ ./manage.py createsuperuser
用者名稱 (leave blank to use 'hoamon'):
電子信箱: hoamon@ho600.com
Password:
Password (again):
Superuser created successfully.
(restful_api_site.py3env) restful_api_site/ $

登入 /admin/ 後,目前只有 2 個 Models (資料表)可以操作:

_images/admin_index.png

「使用者」 Model 頁面

資料表的 4 項基本操作:

  • Create(創建)
  • Read(讀取)
  • Update(更新)
  • Delete(刪除)
_images/user_model.png

在網站開發者的角度上,來說,我們就是在設計「不同介面」來進行這 4 項操作

層級從低至高如下:

  • DB shell
  • Django shell
  • Web page
  • API
  • API over API

列出 User Model 的所有欄位,並包含相關聯的欄位,如: groups, user_permissions

_images/user_form.png

在 Django Admin 模組的頁面中,我們可以使用 superuser 的帳戶操作

  • 創建/讀取/更新/刪除使用者、群組
  • 將使用者加入某一群組
  • 賦與使用者或群組權限
  • 在這個階段, Django 提供的權限模式,只限於規範某個「使用者或群組」對某個「Model」的權限
  • 導入 django-guardian 後,才能達到規範某個「使用者或群組」對某個 Model 內某筆紀錄的權限

在資料操作上, Django ORM(Object-relational mapping) 將 SQL 語法包裝起來,提供 Python class 來操作資料

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
(restful_api_site.py3env) restful_api_site/ $ ./manage.py shell
Python 3.7.5 (default, Dec  8 2019, 11:41:26)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from django.contrib.auth.models import User, Group
In [2]: u = User(username='hoamon', email='hoamon@ho600.com')
In [3]: u.save()
# SQL3: INSERT INTO auth_user (`username`, `email`) VALUES ('hoamon', 'hoamon@ho600.com');
In [4]: User.objects.get(username='hoamon')
# SQL4: SELECT * from auth_user where username = 'hoamon';
In [5]: User.objects.get(username='hoamon').update(last_name='ho')
# SQL5: UPDATE auth_user set last_name = 'ho' where username = 'hoamon';
In [6]: User.objects.get(username='hoamon').delete()
# SQL6: DELETE FROM auth_user where username = 'hoamon';

ORM 的概念就是把 Table 對應成 Model class ,而 Table 中的 1 筆紀錄就是 Model class 實例化後的 object 。

上面的 Django shell ,與預設的 Django shell 長得不一樣,是因為有另外安裝 ipython 套件,安裝方式: pip install ipython

Django 預設給的 User, Group 的可簡單定義如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
class Group(models.Model):
    name = models.CharField(max_length=150, unique=True)
    permissions = models.ManyToManyField(Permission, blank=True)
class User(models.Model):
    username = models.CharField(max_length=150)
    password = models.CharField(max_length=128)
    first_name = models.CharField(max_length=30, blank=True)
    last_name = models.CharField(max_length=150, blank=True)
    email = models.EmailField(blank=True)
    is_active = models.BooleanField(default=False)
    is_staff = models.BooleanField(default=False)
    is_superuser = models.BooleanField(default=False)
    date_joined = models.DateTimeField(auto_now_add=True)
    last_login = models.DateTimeField()
    groups = models.ManyToManyField(Group, blank=True)
    user_permissions = models.ManyToManyField(Permission, blank=True)
_images/tables.png

Permission, Group, User 等 3 個 Model 所對應到的 DB Table

利用 ./manage.py dbshell 進入 MariaDB shell 來觀看它們的結構

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
(restful_api_site.py3env) restful_api_site/ $ ./manage.py dbshell
MariaDB [restful_api_site]> show create table auth_group;
+------------+--------------------------------------------------------------------+
| Table      | Create Table                                                       |
| auth_group | CREATE TABLE `auth_group` (                                        |
|            |    `id` int(11) NOT NULL AUTO_INCREMENT,                           |
|            |    `name` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,        |
|            |    PRIMARY KEY (`id`),                                             |
|            |    UNIQUE KEY `name` (`name`)                                      |
|            | ) ENGINE=InnoDB AUTO_INCREMENT=2                                   |
|            |   DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci               |
+------------+--------------------------------------------------------------------+
1 row in set (0.010 sec)
  1. 創建 2 個使用者及 1 個群組
  2. 將 2 個使用者都加入這個群組
  3. 刪除其中 1 個使用者
  4. 列出群組中的使用者
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
In [1]: from django.contrib.auth.models import User, Group
In [2]: u1 = User(username='user1', email='user1@ho600.com')
In [3]: u1.save()
In [4]: u2 = User(username='user2', email='user2@ho600.com')
In [5]: u2.save()
In [6]: g1 = Group(name='Normal User')
In [7]: g1.save()
In [8]: u1.groups.add(g1)
In [9]: g1.user_set.add(u2)
In [10]: for u in User.objects.all().order_by('id')[:2]:
    ...:     print("{}, {}".format(u.id, u.username))
1, user1
2, user2
In [11]: from django.db.models import Q
In [12]: for u in g1.user_set.all().filter(
    ...:     username__in=['user1', 'user2']
    ...:     ).filter(Q(id=1, username='user1')
    ...:              |Q(id=2, username='user2')
    ...:             ).order_by('-id'):
    ...:     print(u.username)
2 user2
1 user1
In [13]: u2.delete()
In [14]: for u in g1.user_set.filter(username__isnull=False):
    ...:     print(u.username)
user1

Model 設計

從 MQTT Subscriber 函式所傳來的資料格式,可能如下:

欄位 說明
topic ho600/office/power1 Iot 感測器登記的代號
timestamp 1579262426.123045 感測器紀錄的時間,以 unix timestamp 格式紀錄
value 23.45 感測值,如: 電流值、溫濕度、亮度

EndSpot 放置感測器的設定,FlowData 則紀錄每一筆感測資料。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
class EndSpot(models.Model):
    topic = models.CharField(max_length=150, unique=True)
    note = models.TextField()

    class Meta:
        permissions = (
            ('add_flowdata_under_this_end_spot',
             'Add FlowData records under This EndSpot'),
        )

class FlowData(models.Model):
    end_spot = models.ForeignKey(EndSpot, on_delete=models.CASCADE)
    timestamp = models.DecimalField(max_digits=20, decimal_places=6, db_index=True)
    value = models.FloatField() #IFNO: in some cases, DecimalField is better
    create_time = models.DateTimeField(auto_now_add=True, db_index=True)

create django app

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
(restful_api_site.py3env) restful_api_site/ $ django-admin startapp data_store
(restful_api_site.py3env) restful_api_site/ $ git add data_store && \
git ci -m "Initial data_store app"
[master c479679] Initial data_store app
7 files changed, 17 insertions(+)
create mode 100644 restful_api_site/data_store/__init__.py
create mode 100644 restful_api_site/data_store/admin.py
create mode 100644 restful_api_site/data_store/apps.py
create mode 100644 restful_api_site/data_store/migrations/__init__.py
create mode 100644 restful_api_site/data_store/models.py
create mode 100644 restful_api_site/data_store/tests.py
create mode 100644 restful_api_site/data_store/views.py

https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/c479679b162ef796835e031a28c2d447d3c16536

  1. 添加 data_store 到 settings.INSTALLED_APPS ( 修改:9006318 )
  2. 把 2 個 Models 定義置入 data_store/models.py ( commit:c6e82a5b )
  3. 執行 ./manage.py makemigrations 以生成 db schema migration 檔 ( commit:945ab91b )
  4. 執行 ./manage.py migrate , Django 會拿上一動作的 migration 檔來調整資料庫中的表架構: 新增表格、新增欄位、新增 Key 、…

執行 migrate 指令時, django 會從 django_migrations table 中,找尋已執行的 migrations file 紀錄:

id app name applied
1 contenttypes 0001_initial 2020-01-17 04:31:16.111321
4 admin 0002_logentry_remove_auto_add 2020-01-17 04:31:16.545302
.  
17 sessions 0001_initial 2020-01-17 04:31:16.812397

在比對出 data_store/migrations/0001_initial.py 的紀錄並未在 django_migrations 中,那就執行 data_store/migrations/0001_initial.py 內的程式:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# data_store/migrations/0001_initial.py
class Migration(migrations.Migration):
    initial = True
    dependencies = [
    ]
    operations = [
        migrations.CreateModel(
            name='EndSpot',
            fields=[
                ('id', models.AutoField(auto_created=True,
                                        primary_key=True,
                                        serialize=False,
                                        verbose_name='ID')),
                ('topic', models.CharField(max_length=150, unique=True)),
                ('note', models.TextField()),
            ],
            options={
                'permissions': (('add_flowdata_under_this_end_spot',
                                'Add FlowData records under This EndSpot'),),
            },
        ),
        migrations.CreateModel(
            name='FlowData',
            fields=[
                ('id', models.AutoField(auto_created=True,
                                        primary_key=True,
                                        serialize=False,
                                        verbose_name='ID')),
                ('timestamp', models.DecimalField(db_index=True,
                                                decimal_places=6,
                                                max_digits=20)),
                ('value', models.FloatField()),
                ('create_time', models.DateTimeField(auto_now_add=True,
                                                    db_index=True)),
                ('end_spot',
                models.ForeignKey(on_delete=django.db.models.deletion.CASCADE,
                                to='data_store.EndSpot')),
            ],
        ),
    ]

執行 migrate 指令的輸出

1
2
3
4
5
(restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, data_store, sessions
Running migrations:
Applying data_store.0001_initial... OK
  • 資料庫結構在升級後,會多了 data_store_endspot, data_store_flowdata 兩張表。
  • 在這個階段要新增紀錄,只有利用 dbshell, shell 指令,以 SQL 或 Python ORM 語法處理。

將 EndSpot, FlowData 登記到 Admin 模組中,修改程式碼( a9fa501 )如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# data_store/admin.py
from django.contrib import admin
from data_store.models import EndSpot, FlowData

class EndSpotAdmin(admin.ModelAdmin):
    pass
admin.site.register(EndSpot, EndSpotAdmin)

class FlowDataAdmin(admin.ModelAdmin):
    pass
admin.site.register(FlowData, FlowDataAdmin)

Django Admin 頁面就能見到 EndSpot, FlowData Models

_images/data_store_two_modeladmins.png

如同 User, Group models ,也可以對 EndSpot, FlowData 作 CRUD 操作

_images/create_end_spot.png

Topic 為必填欄位, Note 則隨意

_images/require_end_spot.png

建立 FlowData 紀錄時, End Spot object 為必填欄位

_images/bad_str.png

在 End Spot 下拉選單中,只秀出 id ,難以辦識

1
2
3
class EndSpot(models.Model):
+    def __str__(self):
+        return self.topic

在 EndSpot Model 中,加入 __str__ 函式,可自定偏好的顯示名稱( 2cc4f64 )。

_images/good_name.png

可顯示 ho600/office/power1

使用 django-restframework 來建立 API 服務

RESTful API Service 賦與 HTTP METHOD 額外的定義:

  • POST => Create
  • GET => Read
  • PATCH/PUT => Update
    • PATCH: 只更新有設定的欄位
    • PUT: 更新所有的欄位,未提供則以空值論
  • DELETE => Delete

http protocol 就是個「明碼文字的傳輸協定」

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ telnet icanhazip.com 80
Trying 104.20.17.242...
Connected to icanhazip.com.
Escape character is '^]'.
GET / HTTP/1.0               <-- I type
Host: icanhazip.com          <-- I type

HTTP/1.1 200 OK
Date: Fri, 17 Jan 2020 17:01:32 GMT
Content-Type: text/plain
Content-Length: 14
Connection: close
Set-Cookie: __cfduid=d1fb84a3f46ea313400cb2c5731f2e88a1579280492; expires=Sun, 16-Feb-20 17:01:32 GMT; path=/; domain=.icanhazip.com; HttpOnly; SameSite=Lax
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
X-RTFM: Learn about this site at http://bit.ly/icanhazip-faq and do not abuse the service.
X-SECURITY: This site DOES NOT distribute malware. Get the facts. https://goo.gl/1FhVpg
X-Worker-Version: 20190626_1
Alt-Svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Server: cloudflare
CF-RAY: 5569e427fcaff065-TPE

92.196.51.109
Connection closed by foreign host.

django-restframework

$ pip install djangorestframework

最好把它登記到 requirements.txt ( 19e1982 )

這樣之後在換地方開發時,直接 pip install -r requirements.txt ,就不會漏安裝它

為 FlowData 生出 GET/POST 的 API endpoint ,只要處理下面 4 個地方:

  • 將 rest_framework 加入 settings.INSTALLED_APPS ( 4c92c72 )
  • 撰寫 FlowDataSerializer ( 061dc7f )
  • 撰寫 FlowDataModelViewSet ( 520ae9e5 )
  • 在 restful_api_site/urls.py 設定 router ( 824cc7a2 )
1
2
3
4
5
6
7
# settings.py

    'django.contrib.staticfiles',

    'data_store',
+    'rest_framework',
]

完成後,即可在 http://127.0.0.1:8000/api/v1/ 看到 BrowsableAPIRenderer 生成出來的 html 網頁:

_images/flow_data_endpoint.png

/api/v1/flowdata/ 的畫面,同時可以看到 objects ,也提供 POST Form

_images/flow_data_endpoint.png

/api/v1/flowdata/ 的畫面,同時可以看到 objects ,也提供 POST Form

_images/flow_data_endpoint_in_json.png

querystring 設定 format=json 後,則只出現 json 格式的所有紀錄

Q: 為什麼只出現 flow_data 的 json ?

先使用 curl 來測試:

1
2
3
4
5
6
7
$ curl -X POST -H "Content-Type: application/json" \
-d '{ "end_spot": 1, "timestamp": "1579283621.327474", "value": 1.4 }' \
'http://127.0.0.1:8000/api/v1/flowdata/?format=json'

{"id":4,"resource_uri":"http://127.0.0.1:8000/api/v1/flowdata/4/?format=json",
"timestamp":"1579283621.327474","value":1.4,
"create_time":"2020-01-17T18:00:40.909966Z","end_spot":1}

可以得到伺服器回傳給我們的新紀錄 id 為 4 。

完成 post_data 函式

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
import requests
def post_data(*args, **kw):
    msg = args[0]
    url = 'http://127.0.0.1:8000/api/v1/flowdata/?format=json'
    topic_mapping = {
        "ho600/office/power1": 1,
    }
    data = {
        "end_spot": topic_mapping[msg.topic],
        "timestamp": msg.payload.get('timestamp', ''),
        "value": msg.payload.get('value', ''),
    }
    res = requests.post(url, data=data)
    print(res.text)
    #INFO: {"id":5,
    #       "resource_uri":
    #           "http://127.0.0.1:8000/api/v1/flowdata/5/?format=json",
    #       "timestamp":"123.123456","value":4.1,
    #       "create_time":"2020-01-17T18:11:42.967727Z","end_spot":1}
感謝大家的聆聽