講者: 何岳峰 Amon Ho
簡報: https://www.ho600.com/presentations/20200118at768/contents.html
文件: https://book-for-import-django.readthedocs.io/zh_TW/latest/01part.html 程式碼: https://github.com/ho600-ltd/examples-of-book-for-import-django
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | import logging
import paho.mqtt.client as mqtt
def post_data(*args, **kw):
"""
How to program this function?
"""
pass
def on_connect(client, userdata, flags, rc):
lg = logging.getLogger('info')
lg.debug("Connected with result code: {}".format(rc))
client.subscribe("ho600/office/power1")
def on_message(client, userdata, msg):
lg = logging.getLogger('info')
lg.debug("{} {}".format(msg.topic, msg.payload))
pos_data(msg)
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("my-iot.domain.com", 1883, 60)
client.loop_forever()
|
目前常見的有 git, mercurial, svn ,而 git 是目前最熱門的
本範例程式碼放在 https://github.com/ho600-ltd/examples-of-book-for-import-django/
scoop for Windows PowerShell: https://scoop.sh/
1 2 3 4 5 6 7 8 9 | PS> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
PS> Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')
PS> scoop install aria2
PS> scoop bucket add extras
PS> Install-Module -Name PowerShellGet -Force Exit # with administrator permission
PS> PowerShellGet\Install-Module posh-git -Scope CurrentUser -AllowPrerelease -Force # with general user
PS> Add-PoshGitToProfile -AllUsers -AllHosts # with administrator permission
PS> # just add a line break
PS> scoop install virtualenv
|
brew for macos: https://brew.sh/
1 2 | $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew install virtualenv
|
apt for Ubuntu: 系統內建
1 2 | $ sudo apt install python3-pip
$ pip3 install virtualenv
|
print('Hello World!')
$ virtualenv -p python2 --no-site-packages py2env
$ source py2env/bin/activate
(py2env) $ pip install "Django>=1,<2"
$ virtualenv -p python3.7 --no-site-packages py37env
$ source py37env/bin/activate
(py37env) $ pip install "Django>=2,<3"
$ virtualenv -p python3.5 --no-site-packages py35env
$ source py35env/bin/activate
(py35env) $ pip install "Django>=2,<3"
1 2 3 4 5 6 7 8 9 | $ virtualenv -p python3 --no-site-packages restful_api_site.py3env
$ source restful_api_site.py3env/bin/activate \
# In PowerShell: \
# PS C:\> restful_api_site.py3env\scripts\activate
(restful_api_site.py3env) $ pip install "Django>2.2,<2.3"
...
Successfully installed Django-2.2.9 sqlparse-0.3.0
(restful_api_site.py3env) $ django-admin startproject restful_api_site
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | (restful_api_site.py3env) $ cd restful_api_site
(restful_api_site.py3env) restful_api_site/ $ ls
manage.py restful_api_site
(restful_api_site.py3env) restful_api_site/ $ git init && git add . && git ci -m '...'
...
[master cc69bfb] ...
5 files changed, 126 insertions(+)
create mode 100644 ...
(restful_api_site.py3env) restful_api_site/ $ git di cc69bfb^..cc69bfb --name-only
restful_api_site/manage.py
restful_api_site/restful_api_site/__init__.py
restful_api_site/restful_api_site/settings.py
restful_api_site/restful_api_site/urls.py
restful_api_site/restful_api_site/wsgi.py
|
restful_api_site 專案從無到初始化的程式碼差異比對: https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/cc69bfb9
(restful_api_site.py3env) restful_api_site/ $ git di cc69bfb^..cc69bfb restful_api_site/settings.py
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | diff --git a/restful_api_site/restful_api_site/settings.py b/restful_api_site/restful_api_site/settings.py
new file mode 100644
index 0000000..5a8707d
--- /dev/null
+++ b/restful_api_site/restful_api_site/settings.py
@@ -0,0 +1,120 @@
+"""
+Django settings for restful_api_site project.
+Generated by 'django-admin startproject' using Django 2.2.9.
...
+"""
...
+ROOT_URLCONF = 'restful_api_site.urls'
...
+WSGI_APPLICATION = 'restful_api_site.wsgi.application'
+DATABASES = { 'default': {
+ 'ENGINE': 'django.db.backends.sqlite3',
+ 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), }}
...
+LANGUAGE_CODE = 'en-us'
+TIME_ZONE = 'UTC'
+USE_I18N = True
+USE_L10N = True
+USE_TZ = True
+STATIC_URL = '/static/'
|
此修改版本的 settings.py 內容可到 https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/cc69bfb98c16ffcb72d36dd15f23d0ccb72d9cf4#diff-21a51302934f25442a0bd16766f498df
在目前這個階段, restful_api_site 是一個擁有 django 預設功能的網站,而資料庫管理系統上預設是用 sqlite3 ,其設定方式在 settings.py :
1 2 3 4 5 6 | DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
|
1 2 3 4 5 6 7 8 9 10 11 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
...
Applying contenttypes.0002_remove_content_type_name... OK
...
Applying sessions.0001_initial... OK
|
1 2 3 4 5 6 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py runserver
...
January 17, 2020 - 03:35:27
Django version 2.2.9, using settings 'restful_api_site.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
|
因為 settings.LANGUAGE_CODE = “en-us” ,所以網頁是英文的
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | DATABASES = {
'default': {
- 'ENGINE': 'django.db.backends.sqlite3',
- 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
+ 'ENGINE': 'django.db.backends.mysql',
+ 'NAME': 'restful_api_site',
+ 'USER': 'restful_api_site',
+ 'PASSWORD': 'restful_api_site_pw', # os.env['RESTFUL_API_SITE_PW']
+ 'HOST': 'my.mariadb.host',
+ 'PORT': '3306',
+ 'OPTIONS': {
+ },
}
}
...
-LANGUAGE_CODE = 'en-us'
+LANGUAGE_CODE = 'zh-Hant'
|
https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/ca533439e
1 2 3 4 5 6 | $ mysql -h my.mariadb.host -u root -p
MariaDB [(none)]> CREATE DATABASE restful_api_site
CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
MariaDB [(none)]> create user 'restful_api_site'@'%' identified by 'restful_api_site_pw';
MariaDB [(none)]> GRANT ALL PRIVILEGES on restful_api_site.* to restful_api_site@'%';
MariaDB [(none)]> \q
|
先紀錄到 restful_api_site/requirements.txt ,再安裝
1 2 3 | # requirements.txt
Django>=2.2,<2.3
mysqlclient==1.4.5
|
1 2 3 4 5 6 | (restful_api_site.py3env) restful_api_site/ $ pip install -r requirements.txt
...
Successfully installed mysqlclient-1.4.5
(restful_api_site.py3env) restful_api_site/ $ \
ls ../restful_api_site.py3env/lib/python3.7/site-packages/mysqlclient-1.4.5.dist-info
INSTALLER LICENSE METADATA RECORD WHEEL top_level.txt
|
1 2 3 4 5 6 7 8 9 10 11 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
...
Applying contenttypes.0002_remove_content_type_name... OK
...
Applying sessions.0001_initial... OK
|
到本階段為止,範例程式碼的進度在 https://github.com/ho600-ltd/examples-of-book-for-import-django/commit/76c5dd8180e115124c26e0c911dfae27670749b1
預設的 urls.py 有列入 Django Admin 模組的進入網址
1 2 3 4 5 | from django.contrib import admin
from django.urls import path
urlpatterns = [
path('admin/', admin.site.urls),
]
|
在 http://127.0.0.1:8000/admin/ ,可以看見一個登入頁:
1 2 3 4 5 6 7 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py createsuperuser
用者名稱 (leave blank to use 'hoamon'):
電子信箱: hoamon@ho600.com
Password:
Password (again):
Superuser created successfully.
(restful_api_site.py3env) restful_api_site/ $
|
登入 /admin/ 後,目前只有 2 個 Models (資料表)可以操作:
資料表的 4 項基本操作:
層級從低至高如下:
在資料操作上, Django ORM(Object-relational mapping) 將 SQL 語法包裝起來,提供 Python class 來操作資料
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py shell
Python 3.7.5 (default, Dec 8 2019, 11:41:26)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from django.contrib.auth.models import User, Group
In [2]: u = User(username='hoamon', email='hoamon@ho600.com')
In [3]: u.save()
# SQL3: INSERT INTO auth_user (`username`, `email`) VALUES ('hoamon', 'hoamon@ho600.com');
In [4]: User.objects.get(username='hoamon')
# SQL4: SELECT * from auth_user where username = 'hoamon';
In [5]: User.objects.get(username='hoamon').update(last_name='ho')
# SQL5: UPDATE auth_user set last_name = 'ho' where username = 'hoamon';
In [6]: User.objects.get(username='hoamon').delete()
# SQL6: DELETE FROM auth_user where username = 'hoamon';
|
ORM 的概念就是把 Table 對應成 Model class ,而 Table 中的 1 筆紀錄就是 Model class 實例化後的 object 。
上面的 Django shell ,與預設的 Django shell 長得不一樣,是因為有另外安裝 ipython 套件,安裝方式: pip install ipython
Django 預設給的 User, Group 的可簡單定義如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | class Group(models.Model):
name = models.CharField(max_length=150, unique=True)
permissions = models.ManyToManyField(Permission, blank=True)
class User(models.Model):
username = models.CharField(max_length=150)
password = models.CharField(max_length=128)
first_name = models.CharField(max_length=30, blank=True)
last_name = models.CharField(max_length=150, blank=True)
email = models.EmailField(blank=True)
is_active = models.BooleanField(default=False)
is_staff = models.BooleanField(default=False)
is_superuser = models.BooleanField(default=False)
date_joined = models.DateTimeField(auto_now_add=True)
last_login = models.DateTimeField()
groups = models.ManyToManyField(Group, blank=True)
user_permissions = models.ManyToManyField(Permission, blank=True)
|
Permission, Group, User 等 3 個 Model 所對應到的 DB Table
利用 ./manage.py dbshell 進入 MariaDB shell 來觀看它們的結構
1 2 3 4 5 6 7 8 9 10 11 12 13 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py dbshell
MariaDB [restful_api_site]> show create table auth_group;
+------------+--------------------------------------------------------------------+
| Table | Create Table |
| auth_group | CREATE TABLE `auth_group` ( |
| | `id` int(11) NOT NULL AUTO_INCREMENT, |
| | `name` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL, |
| | PRIMARY KEY (`id`), |
| | UNIQUE KEY `name` (`name`) |
| | ) ENGINE=InnoDB AUTO_INCREMENT=2 |
| | DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci |
+------------+--------------------------------------------------------------------+
1 row in set (0.010 sec)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | In [1]: from django.contrib.auth.models import User, Group
In [2]: u1 = User(username='user1', email='user1@ho600.com')
In [3]: u1.save()
In [4]: u2 = User(username='user2', email='user2@ho600.com')
In [5]: u2.save()
In [6]: g1 = Group(name='Normal User')
In [7]: g1.save()
In [8]: u1.groups.add(g1)
In [9]: g1.user_set.add(u2)
In [10]: for u in User.objects.all().order_by('id')[:2]:
...: print("{}, {}".format(u.id, u.username))
1, user1
2, user2
In [11]: from django.db.models import Q
In [12]: for u in g1.user_set.all().filter(
...: username__in=['user1', 'user2']
...: ).filter(Q(id=1, username='user1')
...: |Q(id=2, username='user2')
...: ).order_by('-id'):
...: print(u.username)
2 user2
1 user1
In [13]: u2.delete()
In [14]: for u in g1.user_set.filter(username__isnull=False):
...: print(u.username)
user1
|
從 MQTT Subscriber 函式所傳來的資料格式,可能如下:
欄位 | 值 | 說明 |
---|---|---|
topic | ho600/office/power1 | Iot 感測器登記的代號 |
timestamp | 1579262426.123045 | 感測器紀錄的時間,以 unix timestamp 格式紀錄 |
value | 23.45 | 感測值,如: 電流值、溫濕度、亮度 |
EndSpot 放置感測器的設定,FlowData 則紀錄每一筆感測資料。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | class EndSpot(models.Model):
topic = models.CharField(max_length=150, unique=True)
note = models.TextField()
class Meta:
permissions = (
('add_flowdata_under_this_end_spot',
'Add FlowData records under This EndSpot'),
)
class FlowData(models.Model):
end_spot = models.ForeignKey(EndSpot, on_delete=models.CASCADE)
timestamp = models.DecimalField(max_digits=20, decimal_places=6, db_index=True)
value = models.FloatField() #IFNO: in some cases, DecimalField is better
create_time = models.DateTimeField(auto_now_add=True, db_index=True)
|
1 2 3 4 5 6 7 8 9 10 11 12 | (restful_api_site.py3env) restful_api_site/ $ django-admin startapp data_store
(restful_api_site.py3env) restful_api_site/ $ git add data_store && \
git ci -m "Initial data_store app"
[master c479679] Initial data_store app
7 files changed, 17 insertions(+)
create mode 100644 restful_api_site/data_store/__init__.py
create mode 100644 restful_api_site/data_store/admin.py
create mode 100644 restful_api_site/data_store/apps.py
create mode 100644 restful_api_site/data_store/migrations/__init__.py
create mode 100644 restful_api_site/data_store/models.py
create mode 100644 restful_api_site/data_store/tests.py
create mode 100644 restful_api_site/data_store/views.py
|
執行 migrate 指令時, django 會從 django_migrations table 中,找尋已執行的 migrations file 紀錄:
id | app | name | applied |
1 | contenttypes | 0001_initial | 2020-01-17 04:31:16.111321 |
4 | admin | 0002_logentry_remove_auto_add | 2020-01-17 04:31:16.545302 |
. | … | … | |
17 | sessions | 0001_initial | 2020-01-17 04:31:16.812397 |
在比對出 data_store/migrations/0001_initial.py 的紀錄並未在 django_migrations 中,那就執行 data_store/migrations/0001_initial.py 內的程式:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # data_store/migrations/0001_initial.py
class Migration(migrations.Migration):
initial = True
dependencies = [
]
operations = [
migrations.CreateModel(
name='EndSpot',
fields=[
('id', models.AutoField(auto_created=True,
primary_key=True,
serialize=False,
verbose_name='ID')),
('topic', models.CharField(max_length=150, unique=True)),
('note', models.TextField()),
],
options={
'permissions': (('add_flowdata_under_this_end_spot',
'Add FlowData records under This EndSpot'),),
},
),
migrations.CreateModel(
name='FlowData',
fields=[
('id', models.AutoField(auto_created=True,
primary_key=True,
serialize=False,
verbose_name='ID')),
('timestamp', models.DecimalField(db_index=True,
decimal_places=6,
max_digits=20)),
('value', models.FloatField()),
('create_time', models.DateTimeField(auto_now_add=True,
db_index=True)),
('end_spot',
models.ForeignKey(on_delete=django.db.models.deletion.CASCADE,
to='data_store.EndSpot')),
],
),
]
|
執行 migrate 指令的輸出
1 2 3 4 5 | (restful_api_site.py3env) restful_api_site/ $ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, data_store, sessions
Running migrations:
Applying data_store.0001_initial... OK
|
將 EndSpot, FlowData 登記到 Admin 模組中,修改程式碼( a9fa501 )如下:
1 2 3 4 5 6 7 8 9 10 11 | # data_store/admin.py
from django.contrib import admin
from data_store.models import EndSpot, FlowData
class EndSpotAdmin(admin.ModelAdmin):
pass
admin.site.register(EndSpot, EndSpotAdmin)
class FlowDataAdmin(admin.ModelAdmin):
pass
admin.site.register(FlowData, FlowDataAdmin)
|
Django Admin 頁面就能見到 EndSpot, FlowData Models
1 2 3 | class EndSpot(models.Model):
+ def __str__(self):
+ return self.topic
|
在 EndSpot Model 中,加入 __str__ 函式,可自定偏好的顯示名稱( 2cc4f64 )。
使用 django-restframework 來建立 API 服務
RESTful API Service 賦與 HTTP METHOD 額外的定義:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | $ telnet icanhazip.com 80
Trying 104.20.17.242...
Connected to icanhazip.com.
Escape character is '^]'.
GET / HTTP/1.0 <-- I type
Host: icanhazip.com <-- I type
HTTP/1.1 200 OK
Date: Fri, 17 Jan 2020 17:01:32 GMT
Content-Type: text/plain
Content-Length: 14
Connection: close
Set-Cookie: __cfduid=d1fb84a3f46ea313400cb2c5731f2e88a1579280492; expires=Sun, 16-Feb-20 17:01:32 GMT; path=/; domain=.icanhazip.com; HttpOnly; SameSite=Lax
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
X-RTFM: Learn about this site at http://bit.ly/icanhazip-faq and do not abuse the service.
X-SECURITY: This site DOES NOT distribute malware. Get the facts. https://goo.gl/1FhVpg
X-Worker-Version: 20190626_1
Alt-Svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
Server: cloudflare
CF-RAY: 5569e427fcaff065-TPE
92.196.51.109
Connection closed by foreign host.
|
$ pip install djangorestframework
最好把它登記到 requirements.txt ( 19e1982 )
這樣之後在換地方開發時,直接 pip install -r requirements.txt ,就不會漏安裝它
為 FlowData 生出 GET/POST 的 API endpoint ,只要處理下面 4 個地方:
1 2 3 4 5 6 7 | # settings.py
'django.contrib.staticfiles',
'data_store',
+ 'rest_framework',
]
|
完成後,即可在 http://127.0.0.1:8000/api/v1/ 看到 BrowsableAPIRenderer 生成出來的 html 網頁:
先使用 curl 來測試:
1 2 3 4 5 6 7 | $ curl -X POST -H "Content-Type: application/json" \
-d '{ "end_spot": 1, "timestamp": "1579283621.327474", "value": 1.4 }' \
'http://127.0.0.1:8000/api/v1/flowdata/?format=json'
{"id":4,"resource_uri":"http://127.0.0.1:8000/api/v1/flowdata/4/?format=json",
"timestamp":"1579283621.327474","value":1.4,
"create_time":"2020-01-17T18:00:40.909966Z","end_spot":1}
|
可以得到伺服器回傳給我們的新紀錄 id 為 4 。
完成 post_data 函式
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | import requests
def post_data(*args, **kw):
msg = args[0]
url = 'http://127.0.0.1:8000/api/v1/flowdata/?format=json'
topic_mapping = {
"ho600/office/power1": 1,
}
data = {
"end_spot": topic_mapping[msg.topic],
"timestamp": msg.payload.get('timestamp', ''),
"value": msg.payload.get('value', ''),
}
res = requests.post(url, data=data)
print(res.text)
#INFO: {"id":5,
# "resource_uri":
# "http://127.0.0.1:8000/api/v1/flowdata/5/?format=json",
# "timestamp":"123.123456","value":4.1,
# "create_time":"2020-01-17T18:11:42.967727Z","end_spot":1}
|