Programming w/Python and Django
Syllabus
Expand All
|
Contract All
Python
Before Day 1: Level Set
What background do most students have?
Bring all students up to the same level
Setup, Getting Started
Pieces of the Pie
Python Language
Standard Library
Other libraries, modules, frameworks (like Django), etc.
FOSS (Free and Open Source Software)
Free Software Foundation, GNU, Linux, Apache, Python, Django, etc
Google
StackOverflow
Once you have a basic understanding, you can find code to do almost anything, already written and ready for you to use/enhance/contribute
Resources:
https://python.org
https://python.org/help
https://python.org/search
https://docs.python.org/tutorial
https://docs.python.org/reference
https://docs.python.org/library
Built-in Functions = https://docs.python.org/library/functions.html
modules = https://docs.python.org/py-modindex.html
index = https://docs.python.org/genindex.html
Search = https://docs.python.org/search.html
https://docs.python.org/glossary.html
http://rgruet.free.fr/PQR24/PQR2.4.html
Excellent summary card
https://wiki.python.org
https://wiki.python.org/moin/BeginnersGuide/NonProgrammers
https://wiki.python.org/moin/BeginnersGuide/Programmers
https://wiki.python.org/moin/LanguageComparisons
https://wiki.python.org/moin/BeginnersGuide/Download
Community/Support
https://www.python.org/community/lists/
https://mail.python.org/mailman/listinfo/python-list
https://docs.python.org/using
Setup instructions
Command line (Windows, Mac, Unix, or Linux)
http://tutorial.djangogirls.org/en/intro_to_command_line/
Install Python
https://www.python.org/downloads/release/python-2711/
https://developers.google.com/edu/python/set-up
2 Scoops Ch 2
http://tutorial.djangogirls.org/en/installation/
https://docs.python.org/using
Run Python shell (REPL)
% python
% idle
http://pythontutor.com/
Install virtualenv and pip
Create virtual environments
pip freeze > requirements.txt
pip install -r requirements.txt
Install PyCharm
Use PyCharm
Create new file template
from __future__ import unicode_literals
from __future__ import print_function
from __future__ import division
Preferences
Set tabs to 4 spaces
Show margin 80 chars
Show method separators
etc
Pythonic Thinking
Guido van Rossum
DRY
There's one right (and obvious) way to do it
>>> import this
Batteries Included
https://www.python.org/dev/peps/pep-0206/
https://en.wikipedia.org/wiki/Python_(programming_language)
Python Basics
*.py and *.pyc files
Virtual machine (VM), interpreter
Bytecode
Variables
Indentifiers
Case-sensitive
No type specified
_underscore variables
Not imported by import *
Don't call them or override them
__double_underscore variables
Class private name mangling
Don't call them or override them
__special_name_double_underscore__ variables
Override them, don't call them
Special names:
https://docs.python.org/reference/datamodel.html#specialnames
http://stackoverflow.com/questions/1301346/the-meaning-of-a-single-and-a-double-underscore-before-an-object-name-in-python
Assignment
Simple assignment
Assigning multiple values
Chaining assignment
+=, -=, *=, /=, etc.
Print
Store program in a *.py file
Run program from a file
Debug with PyCharm
Set breakpoint
Step Over, Step Into, Resume
Hover
Local variables
Watch
Debug with Python Tutor
http://pythontutor.com/
Arithmetic operators
Unary +, -
+, -, *, /, //, %, **, pow(), abs(), int(), long(), float(), divmod()
Boolean
True, False
and, or, not
Short circuit
bool()
Bitwise
~, ^, &, |, <<, >>
Strings and Numbers
Strings
Use "" or '' for single-line strings
Use """ or ''' for multi-line strings
Adjacent strings are concatenated
Escapes: \n, \r, \t, \xnn, ...
Strings are immutable lists
Numbers
Integer (aka Long): 12, 012, 0x12
Unlimiited precision
Float
Decimal
Complex
x.real, x.imag
Integer division
Addition vs concatenation
Duck typing
Casting
time module
datetime module
None
Expressions
Operator precedence
Parentheses
Truthiness
https://docs.python.org/library/stdtypes.html#truth-value-testing
Control statements
Indentation
If statement, comparisons
if, elif, else
For loop
for x in list1
for x in string1
for x in range(start, end, step)
for, break, continue, else
While loop
while, break, continue, else
Pass
Coding Style
PEP 8
4 spaces, not tabs
TitleCase, camelCase, snake_case
Backslash to escape end of line
Other PEP's (Python Enhancement Proposals)
https://www.python.org/dev/peps/
PEP 20 -- The Zen of Python
Other operators
Input
Basic Containers: Lists and Dicts
Lists
Dicts
Nested Lists and Dicts
Indexing/Slicing Lists/Strings
0-based
x[start:end+1:step]
slice(start, end, step)
Query
in, not in
min(), max()
Adding items to Lists/Strings
+ (concatenate)
* (repeat)
Adding entries to Dicts
Traversing Lists
Traversing Dicts
Traversing Strings
Traversing Slices of Lists/Strings
Iterable
Length of List/String
len()
Removing items from Lists
del
Removing from Dicts
del
Combining Lists/Strings
+ (concatenate)
String methods
chr(), ord(), str()
contains(), startswith(), endswith(), find(), rfind(), index(), rindex(), count()
trim(), strip(), lstrip(), rstrip()
ljust(), rjust(), center(), zfill()
ialnum(), isalpha(), isdigit(), islower(), isupper(), isspace(), istitle()
upper(), lower(), capitalize(), swapcase()
expandtabs()
reversed(), sorted()
replace(), translate(), substitute()
format()
%[flag][width][.precision]s
%[flag][width][.precision]d
%[flag][width][.precision]f
etc.
parse()
split(), rsplit(), splitlines()
join()
decode(), encode()
List methods/operators
list()
list1[start:end+1:step] = list2
append(list)
extend(item)
count(item)
index()
insert()
remove()
pop()
reverse()
sort()
(tuple1, tuple2, tuple3, tuple4) = zip(seq1, seq2, seq3, ...)
Dict methods/operators
dict()
in
len()
dict1[key]
dict1[key] = value
del dict1[key]
clear()
copy(), deepcopy()
has_key(), items(), keys(), values()
iteritems(), iterkeys(), itervalues()
update()
get(key, default), setdefault(key, default)
pop(key, default), popitem()
List Comprehensions
Dict Comprehensions
Set Comprehensions
Functions
Declaring Functions
Parameters
Positional parameters
Immutable objects passed by value
Mutable objects passed by reference
Immutable strings
Default values
All parameters w/o default values first
Then any parameters w/default values
Mutable
Changes persist to subsequent calls
Initialized at declaration not call
Return values
Multiple return values
Default return value
Doc strings
pydoc
Positional arguments
Keyword arguments
Calling functions
Default argument values
Variable argument lists
*args
Declaring *args
"*" is special. "args" is just a convention
Receiving *args (all undeclared positional args)
Passing *args (any list/tuple)
Unpacking *args (as a list)
Passing on *args (modified or unmodified)
**kwargs
Declaring **kwargs
"**" is special. "kwargs" is just a convention
Receiving **kwargs (all undeclared keyword args)
Passing **kwargs (any dict)
Unpacking **kwargs (as a dict)
Passing on **kwargs (modified or unmodified)
Classes
Classes
class Class1(ParentClass1)
Properties
Methods
self parameter ("self" is convention only)
@staticmethod
@classmethod
@property
Object-Oriented Programming
Inheritance from parent class
Mix-ins from multiple parent classes
__init__()
__del__()
super()
Exceptions
Predefined exceptions
Declaring exceptions
Raising exceptions
Catching exceptions
Try
Except
Finally
Else
Re-raising exceptions
assert
Modules
Standard modules
sys, os, os.path, shutil, stat, readline, tempfile, termios, tty
string, re, StringIO
math, random
getopt, optparse, shlex
time, datetime, calendar
platform, commands, glob, mmap, msvcrt, popen2, syslog
base64, binhex, crypt, bz2, gzip, md5, tarfile, zipfile, zipimport, zlib
sets, collections, itertools
imageop, imghdr
sndhdr, sunau, sunaudio, wave, winsound
socket, SocketServer
email, poplib, imaplib, smtplib, smtpd, mailbox, mailcap, mhlib
ftplib, gopherlib, nntplib, telnetlib
xmllib, xml.dom, xml.sax, xmlrpclib
htmllib, htmlentitydefs, markupbase, HTMLParser
urllib, urllib2, urlparse, httplib, webbrowser
SimpleHTTPServer, SimpleXMLRPCServer, Cookie
gettext (internationalization)
thread, threading
inspect, token, tokenize, keyword, symbol, symtable, operator, parser, types, pprint, py_compile
pdb, profile, pstats, trace, traceback
pydoc
test, unittest
etc.
Other modules
PyPI (Python Package Index)
Install via pip
Import
import module1
from module1 import name1
from module1 import *
import module1 as alias1
Use imported code
module.name
name
Creating your own modules
Module hierarchy
Create a single-file module
Create a multi-file module
__init__.py
Typically empty
Can contain code
Code executes during import
Preventing execution on import
Global variables
Decorators
Terminal I/O
input(), raw_input()
print()
File I/O
f = open(name)
f.close()
f.read(), f.readline(), f.readlines(), for line in f
f.write(), f.writelines(),
print(..., f=...) -- prints trailing newlines
f.flush(), f.truncate()
f.seek(), f.tell()
for line in f:
pathlib
os.path
shutil
csv
Advanced Containers: Tuples, Named Tuples, Sets
Set methods/operators
len()
in, not in
for item in set:
add(), remove(), clear(), update()
copy()
issubset(), issuperset()
&, intersection()
|, union()
-, difference()
^, symmetric_difference()
Automated tests
Agile
TDD
Doctests
Unit tests
- Python Assertions and methods
assertEqual, assertNotEqual
assertTrue, assertFalse
assertIs, assertIsNot
assertIsNone, assertIsNotNone
assertIn, assertNotIn
assertIsInstance, assertNotIsInstance
assertRaises, assertRaisesRegexp
assertAlmostEqual, assertNotAlmostEqual
assertGreater, assertGreaterEqual, assertLess, assertLessEqual
assertRegexpMatches, assertNotRegexpMatches,
assertItemsEqual, assertDictContainsSubset
maxDiff
Setup/TearDown
setUp(), tearDown()
setUpClass(), tearDownClass()
setUpModule(), tearDownModule()
More Resources
Docs
Guide to the Python Standard Library
http://www.effbot.org/zone/librarybook-index.htm
3rd party packages
PyPI = Python Package Index
https://www.python.org/pypi/
Code Snippets
http://code.activestate.com/recipes/langs/python
Django
Resources
https://docs.djangoproject.com/en/dev/
Setup
Install Django framework via pip
Install MySQL (or use SQLite for now?)
http://tutorial.djangogirls.org/en/django_installation/
Project Layout
2 Scoops Ch 3
% django-admin.py startproject project1
% python manage.py startapp app1
Your First Django App
Run via PyCharm
Settings and Requirements Files
2 Scoops Ch 4
HTTP Request/Response
http://docs.djangoproject.com
HttpRequest Methods:
method, GET, POST, COOKIES, REQUEST, META
is_ajax(), is_secure()
path, path_info
session
user
etc.
HttpResponse Methods:
write(), flush()
has_header()
etc.
urls.py
http://docs.djangoproject.com
http://django-marcador.keimlink.de/en/urls_views.html#configure-the-urls
View
http://docs.djangoproject.com
http://django-marcador.keimlink.de/en/urls_views.html#add-the-views
redirect()
render()
reverse()
Template
http://docs.djangoproject.com
HTML
http://tutorial.djangogirls.org/en/html/
https://www.codecademy.com/learn/web
Beyond HTML
Expressions {{ expression }}
Tags: {% tag %}
Filters: {{ expression|filter }}
Inheritance
http://tutorial.djangogirls.org/en/template_extending/
{% extends "base.html" %}
{% block <name> %} ... {%endblock %}
Loading tags
{% load %}
Including other templates
{% include "other_template.html" %}
Tags
Predefined Tags
comment/endcomment
if/elif/else/endif , for/endfor
url
Custom Tags
Filters
Predefined Filters
add, divisibleby,
length length_is
cut, slice, join
date, time, floatformat, phone2numeric, timesince, timeuntil
lower, upper, capfirst, pluralize, stringformat
center, ljust, rjust
truncatechars, truncatechars_html, truncatewords, truncatewords_html, wordcount, wordwrap
safe, escape, escapejs, force_escape, fix_ampersands, striptags, removetags, urlencode, etc.
urlize, urlizetrunc
etc.
Custom Filters
CSS
http://tutorial.djangogirls.org/en/css/
Debugging missing pieces
http://tutorial.djangogirls.org/en/extend_your_application/
Model
http://docs.djangoproject.com
Fields
Field Types
CharField, IntegerField, FloatField, DecimalField
BooleanField
DateField, TimeField, DateTimeField, etc.
ChoiceField, MultipleChoiceField, etc.
EmailField, URLField, IPAddressField
FileField, ImageField
Field Options
label, initial, required, widget, validators, etc.
Widgets
TextInput, Textarea, PasswordInput
CheckboxInput, RadioSelect, CheckboxSelectMultiple, NullBooleanSelect
DateInput, TimeInput, DateTimeInput, SelectDateWidget, etc.
Select, SelectMultiple
FileInput, ClearableFileInput
HiddenInput, MultipleHiddenInput
MultiWidget
Meta class
QuerySet
http://docs.djangoproject.com
http://tutorial.djangogirls.org/en/django_orm/
Methods
all()
get()
filter()
get_or_create()
count()
create()
update()
delete()
distinct()
exclude()
order_by()
etc.
Lookups
gt, gte, lt, lte
range
isnull
in, contains, exact, startswith, endswith
icontains, iexact, istartswith, iendswith
etc.
Aggregation
Count(), Sum(), Max(), Min(), Avg(), etc.
Chaining querysets
Q()
Lazy Loading
Form
http://docs.djangoproject.com
http://tutorial.djangogirls.org/en/django_forms/
Defining Forms
Using Forms in Templates
{{ form }}
{{ form.as_table }}
{{ form.as_p }}
{{ form.as_ul }}
{{ form.field1 }}
{{ form.field1.label }}
{{ form.field1.value }}
{{ form.field1.errors }}
Handling Form Input
Validating Data
Validators
Implicit
Explicit
is_valid()
clean_xxx()
clean()
Saving Data
save()
ModelForm
http://docs.djangoproject.com
Polls tutorial
http://docs.djangoproject.com
Debugging
PyCharm
Firebug, Chrome Debugger, etc.
Django Debug Toolbar
Community/Support
Google
StackOverflow
django-users mailing list
Django-users list is very friendly. More than python-list.
IRC (#python, #django)
Meetups
2 Scoops Ch 31
etc.
Login Authentication
@login_required
request.user.is_authenticated()
Django Development Server
Formset
Logging
2 Scoops Ch 24
Fred's logging
Automated Tests
http://docs.djangoproject.com
2 Scoops Ch 20
Running Tests
% python manage.py help test
% python manage.py test
Run all test methods in all test classes of all apps
python manage.py test -v2
More verbose (levels 0-3)
% python manage.py test -v2 appname...
Run all test methods in all test classes of specified apps
% python manage.py test -v3 appname...
Shows ALL steps. Too much detail for typical use
% python manage.py test -v2 appname.TestClass...
Run all test methods in specified test classes
% python manage.py test -v2 appname.TestClass.test_name...
Run specified test methods
Writing Tests
Create tests.py in each app
Inherit from django.test.TestCase
Or django.test.SimpleTestCase if no DB access needed
Django asserts
assertContains, assertNotContains
assertHTMLEqual, assertHTMLNotEqual
assertXMLEqual, assertXMLNotEqual
assertQuerysetEqual
assertRedirects
assertTemplateUsed, assertTemplateNotUsed
assertFormError, assertRaisesMessage
django.test.client
response = self.client.get('/customer/details/')
Test DB
Creates/deletes "test_" DB. Does not touch real DB.
For speed use SQlite, not MySQL
Django suppresses all outgoing email during tests
Debugging tests in PyCharm
Test Coverage
Migrations
http://docs.djangoproject.com
https://helphopelive.atlassian.net/wiki/display/HHLWEB/Migration+Basics
inspectdb
syncdb (before South)
South Migrations (before Django 1.7)
Converting an app to use South instead of syncdb
One-liner
% python manage.py convert_to_south <app_name>
Two-Liner
python manage.py schemamigration <app_name> --initial
% python manage.py migrate <app_name>
Schema migrations
Create/change models
% python manage.py schemamigration <app_name> --auto
Data Migrations
% python manage.py datamigration <app_name> <migration_base_name>
Edit the generated file, adding code to forwards() and backwards()
Migrating forwards or backwards to a specific migration
% python manage.py migrate <app_name> <migration_number>
Migrating all apps to the latest migration for each
% python manage.py migrate --all
Faking a migration (because the change has already been made to the DB)
% python manage.py migrate <app_name> <migration_number> --fake
Django Migrations (Django 1.7 and later)
Same author as South
https://helphopelive.atlassian.net/wiki/display/HHLWEB/Rules+of+Migrations
Security
http://docs.djangoproject.com
2 Scoops Ch 23
Performance
http://docs.djangoproject.com
2 Scoops Ch 22
Middleware
3rd Party Packages
2 Scoops Ch 19
More Resources:
Docs
http://django.readthedocs.io/
http://www.revsys.com/django/cheatsheet/
Tutorials:
http://django-marcador.keimlink.de/en/admin.html
https://github.com/vitorfs/bootcamp
http://www.marinamele.com/taskbuster-django-tutorial
https://tutorial.djangogirls.org/
https://developer.mozilla.org/en-US/docs/Learn/Server-side/Django
3rd party packages
https://www.djangopackages.com/
Code Snippets
https://djangosnippets.org/
Possible Advanced Python Topics
Logging
https://docs.python.org/3/howto/logging.html
Advanced loops
for loop with index also
for index, value in enumerate(list):
http://treyhunner.com/2016/04/how-to-loop-with-indexes-in-python/
for loop over 2 lists at once
for color, ratio in zip(colors, ratios):
http://treyhunner.com/2016/04/how-to-loop-with-indexes-in-python/
iter()
http://treyhunner.com/2016/12/python-iterator-protocol-how-for-loops-work/
iterables, iterators, generators
http://nedbatchelder.com/text/iter.html
hasattr(), getattr(), setattr(), delattr(), dir(), locals(), globals(), vars(), help()
callable(), filter()
exec, execfile(), eval(), compile(), reload()
Python scripts (Linux, Unix, Mac, Windows?)
Shebang for Linux, Unix, Mac
ASSOC and FTYPE for Windows
https://docs.python.org/2/using/windows.html#executing-scripts
Python 3
Why you should use Python 3:
https://eev.ee/blog/2016/07/31/python-faq-why-should-i-use-python-3/
Unicode strings
Set literals
Dict and set comprehensions
Multi-with
print()
collections.OrderedDict
collections.Counter
argparse module improves on optparse
Everything is lazy
Unpacking via * and **
yield from
Keyword-only arguments
Format strings via (f"")
async and await keywords
Function annotations
Matrix multiplication via @
Ellipsis via ...
Enhanced exceptions
__traceback__ and .with_traceback() instead of sys.exc_info()
exception chaining via raise from
super() with no arguments
New metaclass syntax and kwargs for classes
__prepare__
Attribute definition order
__set_name__()
__init_subclass__()
int is long
/ does float division, not integer division, use // for integer division
Other math stuff
Modules shuffled:
StringIO.StringIO --> io.StringIO
SimpleHTTPServer.SimpleHTTPServer --> http.server
pickle/cPickle --> pickle
profile/cProfile --> profile
StringIO/cStringIO --> StringIO
bz2 and gzip improvements
collections.ChainMap
ConfigParser/SafeConfigParser --> ConfigParser
contextlib.ContextDecorator
@contextmanager
contextlib improvements
datetime improvements
functools improvements
inspect improvements
logging improvements
os.scandir is faster
re.fullmatch
shutil.disk_usage()
shutil.get_terminal_size()
subprocess.run()
tempfile.TemporaryDirectory context manager
textwrap.indent()
time improvements
traceback improvements
types improvements
weakref.finalize()
asyncio module
enum module
ipaddress module
pathlib module
selectors module
statistics module
unittest.mock module
faulthandler module
importlib module
tracemalloc module
typing module
venv module
ensurepip module
zipapp module
tab-completion in REPL
__qualname__
Implicit namespace packages
Object finalization improvements
str.format_map()
OSError
ResourceWarning
hasattr() propagates errors better
Hash randomization is on by default
List comprehensions no longer leak their loop variables into the enclosing scope
nonlocal
TypeError during compare
!=
Can access a method as a class attribute
input/raw_input --> input
Speed improvements
http://www.itworld.com/article/3149782/application-development/python-36-is-packed-with-goodness.html
What was removed/changed in Python 3 (how to port from Python 2):
https://eev.ee/blog/2016/07/31/python-faq-how-do-i-port-to-python-3/
Why you should NOT use Python 3:
https://learnpythonthehardway.org/book/nopython3.html
Why you SHOULD used Python 3 after all:
https://eev.ee/blog/2016/11/23/a-rebuttal-for-python-3/
What libraries support Python 3:
http://py3readiness.org/
Python 2 and 3 release notes:
https://docs.python.org/3/whatsnew/
Why Python 3 was created:
http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html
Python features specific to MS Windows:
https://docs.python.org/2/library/windows.html#mswin-specific-services
py2exe
Python 3 on MS Windows
https://docs.python.org/3/using/windows.html
Recursion
Iteration vs Recursion
Context Managers
With statement
RTTI (Run-Time Type Identification)
type()
isinstance()
issubclass()
__name__
__class__
__module__
__file__
im_class, im_self, im_func
Regular Expressions
https://pycon2016.regex.training/
Nested Functions
Anonymous Functions
Closures
Lambdas
The rest of the Python Standard Library
https://docs.python.org/library/
Operator Overriding
Multiple statements on one line
print("foo"); print("bar"); print("semicolon is the separator")
JSON
dict1 = json.loads("String of JSON")
str1 = json.dumps(dict1)
json.dump(dict1, fp=file1)
import json; help(json)
Python Parameter Passing
https://eev.ee/blog/2012/05/23/python-faq-passing/
Possible Advanced Django Topics
Deploying to Production
Apache and collectstatic
2 Scoops Ch 27, 28
Fred's pub script
Heroku
PythonAnywhere
http://tutorial.djangogirls.org/en/deploy/
Advanced ORM
Managers
DB Routers
XML, JSON, SOAP, REST
2 Scoops Ch 14, 15
Class-Based Views
Admin site
http://django-marcador.keimlink.de/en/admin.html
Internationalization
2 Scoops Appendix D
Pagination
Sending email
SEO
Sitemaps
Test Coverage
Performance Profiling
Signals
Possible Related Topics
JIRA -- Bug/feature tracking
Git -- Version control
http://tutorialzine.com/2016/06/learn-git-in-30-minutes/
http://book.git-scm.com/
https://git-scm.com/
Cloud Computing
Jenkins -- Continuous Integration
Docker
2 Scoops Ch 29
Big Data
Map/Reduce
Python's map(), reduce()
Mobile Computing
REST Web Services
(Version 0.6 -- 4/16/2017)