Typeerror: cant subtract offset-naive and offset-aware datetimes

from django.utils import timezone now = timezone.now()

Returns the same as datetime.now(), but always includes timezone data (as long as the USE_TZ setting is True). I would just replace your instances of datetime.now() with this.

As to why "it adds freaking +00:00 at the end", well the object is stored in the database with timezone data by default. If you save an unaware datetime to the database, Django first should warn you about it, then it will assume you want the server's configured timezone set and cast the datetime to that tz when writing to the database. What you retrieve from the database will always by tz-aware for that reason.

$ ceph -v ceph version 15.2.11 (e3523634d9c2227df9af89a4eac33d16738c49cb) octopus (stable) $ rook version rook: v1.5.9 go: go1.13.8 $ ceph mgr module enable orchestrator module 'orchestrator' is already enabled (always-on) $ ceph mgr module enable rook $ ceph orch set backend rook $ ceph mgr module ls { "always_on_modules": [ "balancer", "crash", "devicehealth", "orchestrator", "pg_autoscaler", "progress", "rbd_support", "status", "telemetry", "volumes" ], "enabled_modules": [ "dashboard", "iostat", "prometheus", "restful", "rook" ], ..... $ ceph orch host ls HOST ADDR LABELS STATUS k8s-myhome-master-01 k8s-myhome-master-01 k8s-myhome-master-02 k8s-myhome-master-02 k8s-myhome-node-01 k8s-myhome-node-01 k8s-myhome-node-02 k8s-myhome-node-02 k8s-myhome-node-03 k8s-myhome-node-03 k8s-myhome-node-04 k8s-myhome-node-04 k8s-myhome-node-05 k8s-myhome-node-05 $ ceph orch ls Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1177, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 140, in handle_command return dispatch[cmd['prefix']].call(self, cmd, inbuf) File "/usr/share/ceph/mgr/mgr_module.py", line 318, in call return self.func(mgr, **kwargs) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 102, in <lambda> wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/share/ceph/mgr/orchestrator/module.py", line 547, in _list_services nice_delta(now, s.last_refresh, ' ago'), File "/usr/share/ceph/mgr/orchestrator/module.py", line 28, in nice_delta return to_pretty_timedelta(now - t) + suffix TypeError: can't subtract offset-naive and offset-aware datetimes $ ceph orch ps Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1177, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 140, in handle_command return dispatch[cmd['prefix']].call(self, cmd, inbuf) File "/usr/share/ceph/mgr/mgr_module.py", line 318, in call return self.func(mgr, **kwargs) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 102, in <lambda> wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs) File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/share/ceph/mgr/orchestrator/module.py", line 605, in _list_daemons status += ' (%s)' % to_pretty_timedelta(now - s.started) TypeError: can't subtract offset-naive and offset-aware datetimes

TL;DR: You should always store datetimes in UTC and convert to proper timezone on display.

A timezone offset refers to how many hours the timezone is from Coordinated Universal Time (UTC). The offset of UTC is +00:00, and the offset of Asia/Taipei timezone is UTC+08:00 (you could also present it as GMT+08:00). Basically, there is no perceptible difference between Greenwich Mean Time (GMT) and UTC.

The local time subtracts the offset of its timezone is UTC time. For instance, 18:00+08:00 of Asia/Taipei minuses timezone offset +08:00 is 10:00+00:00, 10 o'clock of UTC. On the other hand, UTC time pluses local timezone offset is local time.

ref:
//opensource.com/article/17/5/understanding-datetime-python-primer
//julien.danjou.info/blog/2015/python-and-timezones

到底是 GMT+8 還是 UTC+8？
//pansci.asia/archives/84978

Installation

$ pip install -U python-dateutil pytz tzlocal

Show System Timezone

import tzlocal tzlocal.get_localzone() # <DstTzInfo 'Asia/Taipei' LMT+8:06:00 STD> tzlocal.get_localzone().zone # 'Asia/Taipei' from time import gmtime, strftime print(strftime("%z", gmtime())) # +0800

ref:
//github.com/regebro/tzlocal
//stackoverflow.com/questions/13218506/how-to-get-system-timezone-setting-and-pass-it-to-pytz-timezone/

Find Timezones Of A Certain Country

import pytz pytz.country_timezones('tw') # ['Asia/Taipei'] pytz.country_timezones('cn') # ['Asia/Shanghai', 'Asia/Urumqi']

ref:
//pythonhosted.org/pytz/#country-information

Offset-naive Datetime

Any naive datetime would be present as local timezone but without tzinfo, so it is buggy.

A naive datetime object contains no timezone information. The datetime_obj.tzinfo will be set to None if the object is naive. Actually, datetime objects without timezone should be considered as a "bug" in your application. It is up for the programmer to keep track of which timezone users are working in.

import datetime import dateutil.parser datetime.datetime.now() # return the current date and time in local timezone, in this example: Asia/Taipei (UTC+08:00) # datetime.datetime(2018, 2, 2, 9, 15, 6, 211358)), naive datetime.datetime.utcnow() # return the current date and time in UTC # datetime.datetime(2018, 2, 2, 1, 15, 6, 211358), naive dateutil.parser.parse('2018-02-04T16:30:00') # datetime.datetime(2018, 2, 4, 16, 30), naive

ref:
//docs.python.org/3/library/datetime.html
//dateutil.readthedocs.io/en/stable/

Offset-aware Datetime

A aware datetime object embeds a timezone information. Rules of thumb for timezone in Python:

Always work with "offset-aware" datetime objects.
Always store datetime in UTC and do timezone conversion only when interacting with users.
Always use ISO 8601 as input and output string format.

There are two useful methods: pytz.utc.localize(naive_dt) for converting naive datetime to timezone be offset-aware, and aware_dt.astimezone(pytz.timezone('Asia/Taipei')) for adjusting timezones of offset-aware objects.

You should avoid naive_dt.astimezone(some_tzinfo) which would be converted to aware datetime as system timezone then convert to some_tzinfo timezone.

import datetime import pytz now_utc = pytz.utc.localize(datetime.datetime.utcnow()) # equals to datetime.datetime.now(pytz.utc) # equals to datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc) # datetime.datetime(2018, 2, 4, 10, 17, 40, 679562, tzinfo=<UTC>), aware now_taipei = now_utc.astimezone(pytz.timezone('Asia/Taipei')) # convert to another timezone # datetime.datetime(2018, 2, 4, 18, 17, 40, 679562, tzinfo=<DstTzInfo 'Asia/Taipei' CST+8:00:00 STD>), aware now_utc.isoformat() # '2018-02-04T10:17:40.679562+00:00' now_taipei.isoformat() # '2018-02-04T18:17:40.679562+08:00' now_utc == now_taipei # True

For working with pytz, it is recommended to call tz.localize(naive_dt) instead of naive_dt.replace(tzinfo=tz). dt.replace(tzinfo=tz) does not handle daylight savings time correctly.

dt1 = datetime.datetime.now(pytz.timezone('Asia/Taipei')) # datetime.datetime(2018, 2, 4, 18, 22, 28, 409332, tzinfo=<DstTzInfo 'Asia/Taipei' CST+8:00:00 STD>), aware dt2 = datetime.datetime(2018, 2, 4, 18, 22, 28, 409332, tzinfo=pytz.timezone('Asia/Taipei')) # datetime.datetime(2018, 2, 4, 18, 22, 28, 409332, tzinfo=<DstTzInfo 'Asia/Taipei' LMT+8:06:00 STD>), aware dt1 == dt2 # False

ref:
//pythonhosted.org/pytz/

Naive and aware datetime objects are not comparable.

naive = datetime.datetime.utcnow() aware = pytz.utc.localize(naive) naive == aware # False naive >= aware # TypeError: can't compare offset-naive and offset-aware datetimes

Parse String to Datetime

python-dateutil usually comes in handy.

import dateutil.parser import dateutil.tz dt1 = dateutil.parser.parse('2018-02-04T19:30:00+08:00') # datetime.datetime(2018, 2, 4, 19, 30, tzinfo=tzoffset(None, 28800)), aware dt2 = dateutil.parser.parse('2018-02-04T11:30:00+00:00') # datetime.datetime(2018, 2, 4, 11, 30, tzinfo=tzutc()), aware dt3 = dateutil.parser.parse('2018-02-04T11:30:00Z') # datetime.datetime(2018, 2, 4, 11, 30, tzinfo=tzutc()), aware dt1 == dt2 == dt3 # True

ref:
//dateutil.readthedocs.io/en/stable/

Convert Datetime To Unix Timestamp

import datetime naive_dt = datetime.datetime(2018, 9, 10, 0, 0, 0) naive_timestamp = aware_dt.timestamp() # naive_dt would be in local timezone, in this example: Asia/Taipei (UTC+08:00) aware_dt = datetime.datetime(2018, 9, 10, 0, 0, 0, tzinfo=datetime.timezone(datetime.timedelta(hours=8))) aware_timestamp = aware_dt.timestamp() naive_timestamp == aware_timestamp # True # MongoDB stores all datetimes in UTC timezone dt_fetched_from_mongodb.replace(tzinfo=datetime.timezone.utc).timestamp()

Parse Unix Timestamp To Datetime

import datetime import time import pytz ts = time.time() # seconds since the Epoch (1970-01-01T00:00:00 in UTC) # 1517748706.063205 dt1 = datetime.datetime.fromtimestamp(ts) # return the date and time of the timestamp in local timezone, in this example: Asia/Taipei (UTC+08:00) # datetime.datetime(2018, 2, 4, 20, 51, 46, 63205), naive dt2 = datetime.datetime.utcfromtimestamp(ts) # return the date and time of the timestamp in UTC timezone # datetime.datetime(2018, 2, 4, 12, 51, 46, 63205), naive pytz.timezone('Asia/Taipei').localize(dt1) == pytz.utc.localize(dt2) # True

ref:
//stackoverflow.com/questions/13890935/does-pythons-time-time-return-the-local-or-utc-timestamp

We might receive an Unix timestamp from a JavaScript client.

var moment = require('moment') var ts = moment('2018-02-02').unix() // 1517500800

ref:
//momentjs.com/docs/#/parsing/unix-timestamp/

Store Datetime In Databases

MySQL lets developers decide what timezone should be used, and you should convert datetime to UTC before saving into database.
MongoDB assumes that all the timestamp are in UTC, and you have to normalize datetime to UTC.

ref:
//tommikaikkonen.github.io/timezones/
//blog.elsdoerfer.name/2008/03/03/fun-with-timezones-in-django-mysql/

Tools

ref:
//www.epochconverter.com/
//www.timeanddate.com/worldclock/converter.html

What is offset naive and offset aware Datetimes?

naive vs aware So a datetime object can be either offset naive or offset aware. A timezone's offset refers to how many hours the timezone is from Coordinated Universal Time (UTC). A naive datetime object contains no timezone information. The easiest way to tell if a datetime object is naive is by checking tzinfo.

How do I remove Tzinfo from datetime?

To remove timestamp, tzinfo has to be set None when calling replace() function. First, create a DateTime object with current time using datetime. now(). The DateTime object was then modified to contain the timezone information as well using the timezone.

What is Tzinfo?

tzinfo is an abstract base class. It cannot be instantiated directly. A concrete subclass has to derive it and implement the methods provided by this abstract class. The instance of the tzinfo class can be passed to the constructors of the datetime and time objects.