Stem Docs

Change Log

Change Log

The following is a log of all user-facing changes to Stem, both released and unreleased. For a monthly report on work being done see my development log.

Versioning

Stem uses semantic versioning, which means that versions consist of three numbers (such as ‘1.2.4’). These are used to convey the kind of backward compatibility you can expect…

  • The first value is the major version. This changes infrequently, and indicates that backward incompatible changes have been made (such as the removal of deprecated functions).

  • The second value is the minor version. This is the most common kind of release, and denotes that the improvements are backward compatible.

  • The third value is the patch version. When a Stem release has a major issue another release is made which fixes just that problem. These do not contain substantial improvements or new features. This value is sometimes left off to indicate all releases with a given major/minor version.

Unreleased

The following are only available within Stem’s git repository.

  • Controller

  • Socket based control connections often raised BrokenPipeError when closed

  • Descriptors

  • transport lines within extrainfo descriptors failed to validate

Version 1.8 (December 29th, 2019)

Stem 1.8 is the final release in Stem’s 1.x series and with it Python 2.x support. Over a year in the making, this introduces CollecTor, bandwidth metric, and HSv3 descriptor support.

  • Controller

  • Added get_start_time() method to the Controller

  • Added get_uptime() method to the Controller

  • Controller events could fail to be delivered in a timely fashion (ticket 27173)

  • Adjusted get_microdescriptors() fallback to also use ‘.new’ cache files (ticket 28508)

  • ExitPolicies could raise TypeError when read concurrently (ticket 29899)

  • Moved the arrived_at attribute from Event to ControlMessage

  • STALE_DESC Flag (spec)

  • DORMANT and ACTIVE Signal (spec)

  • QUERY_RATE_LIMITED HSDescReason (spec)

  • EXTOR and HTTPTUNNEL Listener

  • Descriptors

  • Client

  • Sockets with ORPorts errored if responses exceeded a hardcoded buffer size (ticket 28961)

  • Utilities

  • is_valid_hidden_service_address() now provides true if a v3 hidden servie address

  • Fixed ‘invalid escape sequence’ python 3.6 warnings (ticket 27270)

  • Website

  • Interpreter

  • tor-prompt is now ~34% faster when used to non-interactively invoke commands

Version 1.7 (October 7th, 2018)

Stem 1.7 is a full year of improments. Most notably this adds the ability to download descriptors through ORPorts and the stem.directory module.

  • Controller

  • Listener exceptions and malformed events no longer break further event processing (ticket 27053)

  • Documented v3 hidden service support (ticket 25124, spec)

  • Added the stem.control.MALFORMED_EVENTS event listener constant

  • Added support for limiting the maximum number of streams for create_ephemeral_hidden_service() (spec)

  • Added a timeout argument to Controller methods that could await a response (ticket 26056)

  • Added a close_output argument to launch_tor

  • stem.connection.connect() crashed if its port argument was a string

  • More reliable ExitPolicy resolution (ticket 25739)

  • Fixed cache invalidation when another contorller calls SETCONF (ticket 25821)

  • create_hidden_service() failed when creating services with v2 options (ticket 27446)

  • get_info() commonly raised stem.ProtocolError when it should provide a stem.OperationFailed

  • get_microdescriptors() now reads microdescriptors from the control port rather than disk when available (spec)

  • Added the delivered_read, delivered_written, overhead_read, and overhead_written attributes to CircuitBandwidthEvent (spec)

  • The config attribute of ConfChangedEvent couldn’t represent tor configuration options with multiple values. It has been replaced with new changed and unset attributes.

  • Replaced socket’s get_address(), get_port(), and get_socket_path() with attributes

  • ControlMessage is now comparable and hashable

  • Removed the ‘raw’ argument from send()

  • Descriptors

  • stem.descriptor.remote can now download from relay ORPorts

  • Zstd and lzma compression support (spec)

  • Moved the Directory classes into their own stem.directory module

  • Added from_cache() and from_remote() to the DirectoryAuthority subclass

  • Tor rearranged its files, adjusted stem.descriptor.remote.Directory.from_remote() and stem.manual.Manual.from_remote() to account for this

  • Fallback directory v2 support, which adds nickname and extrainfo

  • Added the orport_v6 attribute to the Authority class

  • Added server descriptor’s new is_hidden_service_dir attribute

  • Added the network status vote’s new bandwidth_file_headers attribute (spec)

  • Added the microdescriptor router status entry’s new or_addresses attribute (ticket 26405, spec)

  • Don’t retry downloading descriptors when we’ve timed out

  • Don’t download from tor26, an authority that frequently timeout

  • Replaced Bifroest bridge authority with Serge (ticket 26771)

  • stem.descriptor.remote now consistently defaults fall_back_to_authority to false

  • Deprecated stem.descriptor.export. If you use it please let us know.

  • Added their_server_descriptor()

  • Added the reply_headers attribute to Query

  • Supplying a User-Agent when downloading descriptors

  • Reduced maximum descriptors fetched by the remote module to match tor’s new limit (ticket 24743)

  • Consensus shared_randomness_*_reveal_count attributes undocumented, and unavailable if retrieved before their corresponding shared_randomness_*_value attribute (ticket 25046)

  • Allow ‘proto’ line to have blank values (spec)

  • Utilities

  • Fixed PyPy compatibility (ticket 26207)

  • Python 3.6+ syntax error if test_tools.py imported (ticket 26739)

  • Connection information from proc limited to 10,000 results

  • Include attribute types in most equality checks and hashes

  • Cache hash values of immutable classes

  • More performant string concatenation via bytearrays

  • Functions using lru_cache could fail with a KeyError on Python 3.5 (ticket 26412)

  • Website

  • Version 1.7.1 (December 26th, 2018) - launch_tor() compatibility with an upcoming log format change (ticket 28731)

Version 1.6 (November 5th, 2017)

Year long accumulation of fixes and improvements in support of the Nyx 2.0 release.

  • Controller

  • launch_tor() raised a ValueError if invoked when outside the main thread

  • Failure to authenticate could raise an improper response or hang (ticket 22679)

  • Renamed ConnectionBandwidthEvent type attribute to conn_type to avoid conflict with parent class (ticket 21774)

  • Added ‘force’ argument to save_conf() (spec)

  • Added the QUERY_NO_HSDIR HSDescReason and recognizing unknown HSDir results (spec)

  • Added the GUARD_WAIT CircStatus (spec)

  • Unable to use cookie auth when path includes wide characters (chinese, japanese, etc)

  • Tor change caused list_ephemeral_hidden_services() to provide empty strings if unset (ticket 21329)

  • Better error message when set_conf() fails due to an option being immutable

  • get_ports() didn’t provide ports for many representations of localhost (ticket 24085)

  • is_geoip_unavailable() now determines if database is available right away

  • Added the time attribute to StreamBwEvent and CircuitBandwidthEvent (spec)

  • Added the consensus_content attribute to NewConsensusEvent and deprecated its ‘desc’

  • Deprecated is_geoip_unavailable(), this is now available via getinfo instead (ticket 23237, spec)

  • Deprecated AuthDirNewDescEvent (ticket 22377, spec)

  • Caching manual information as sqlite rather than stem.util.conf, making stem.manual.Manual.from_cache() about ~8x faster

  • Added database() to get a cursor for the manual cache

  • Failed to parse torrcs without a port on ipv6 exit policy entries

  • Resilient to ‘Tor’ prefix in ‘GETINFO version’ result (spec)

  • Added a all_extra parameter to stem.version.Version and support for multiple parenthetical entries (ticket 22110, spec)

  • Setting ‘UseMicrodescriptors 1’ in your torrc caused get_network_statuses() to error (ticket 24110)

  • Closing controller connection faster when under heavy event load

  • Better messaging when unable to connect to tor on FreeBSD

  • More succinct trace level logging

  • Descriptors

  • Utilities

  • Support connection resolution on OpenBSD using fstat (ticket 13807)

  • Added size_of()

  • Added is_tracing()

  • Added timeout argument to call()

  • Added cwd argument to call()

  • Added round argument to size_label()

  • Added TimedTestRunner and test_runtimes()

  • Supporting pid arguments in is_running()

  • Made connection resolution via proc about 5x faster

  • Normalized format() to return unicode

  • Don’t load vim swap files as configurations

  • Interpreter

  • Website

  • Source code served by ‘[source]’ links perpetually stale (ticket 19521)

Version 1.5 (November 20th, 2016)

Stem 1.5 is a long overdue accumulation of seventeen months of improvements including dramatically improved python 3.x performance, tor manual information, and much more.

  • Controller

  • Dramatic, 300x performance improvement for reading from the control port with python 3

  • Added stem.manual, which provides information available about Tor from its manual (ticket 8251)

  • connect() and from_port() now connect to both port 9051 (relay’s default) and 9151 (Tor Browser’s default) (ticket 16075)

  • ExitPolicy support for accept6/reject6 and *4/6 wildcards (ticket 16053)

  • Added support for NETWORK_LIVENESS events (spec)

  • Added support for basic authentication to create_ephemeral_hidden_service() (spec)

  • Added support for non-anonymous services to create_ephemeral_hidden_service() (spec)

  • Added event_description() for getting human-friendly descriptions of tor events (ticket 19061)

  • Added reconnect() to the Controller

  • Added is_set() to the Controller

  • Added is_user_traffic_allowed() to the Controller

  • Added the replica attribute to HSDescEvent (spec)

  • Added the NoEdConsensus Flag (spec)

  • Recognize listeners with IPv6 addresses in get_listeners()

  • launch_tor() could leave a lingering process during an unexpected exception (ticket 17946)

  • IPv6 addresses could trigger errors in get_listeners(), ORConnEvent, and quite a few other things (ticket 16174)

  • Don’t obscure stacktraces, most notably Controller getter methods with default values

  • Classes with custom equality checks didn’t provide a corresponding inequality method

  • Descriptors

  • Shorthand functions for stem.descriptor.remote

  • Added fallback directory information.

  • Support for ed25519 descriptor fields (spec)

  • Support downloading microdescriptor consensus with :func:~stem.descriptor.remote.DescriptorDownloader.get_consensus` (:spec`e788b8f`)

  • Added consensus and vote’s new shared randomness attributes (spec)

  • Added server descriptor’s new allow_tunneled_dir_requests attribute (spec)

  • Server descriptor validation fails with ‘extra-info-digest line had an invalid value’ from additions in proposal 228 (ticket 16227)

  • BridgeDescriptor now has ‘ntor_onion_key’ like its unsanitized counterparts

  • Replaced the Microdescriptor identifier and identifier_type attributes with an identifiers hash since it can now appear multiple times (spec)

  • Unable to read descriptors from data directories on Windows due to their CRLF newlines (ticket 17051)

  • TypeError under python3 when using ‘use_mirrors = True’ (ticket 17083)

  • Deprecated hidden service descriptor’s introduction_points_auth field, which was never implemented in tor (ticket 15190, spec)

  • Deprecated get_microdescriptors() as it was never implemented in tor (ticket 9271)

  • get_hidden_service_descriptor() errored when provided a servers argument (ticket 18401)

  • Fixed parsing of server descriptor’s allow-single-hop-exits and caches-extra-info lines

  • Bracketed IPv6 addresses were mistreated as being invalid content

  • Better validation for non-ascii descriptor content

  • Updated dannenberg’s v3ident (ticket 17906)

  • Removed urras as a directory authority (ticket 19271)

  • Utilities

  • IPv6 support in get_connections() when resolving with proc, netstat, lsof, or ss (ticket 18079)

  • The ‘ss’ connection resolver didn’t work on Gentoo (ticket 18079)

  • Recognize IPv4-mapped IPv6 addresses in our utils (ticket 18079)

  • Allow stem.util.conf.Config.set() to remove values when provided with a None value

  • Support prefix and suffix issue strings in pyflakes_issues()

  • Additional information when call() fails through a CallError

  • Added stem.util.system.SYSTEM_CALL_TIME with the total time spent on system calls

  • Added an is_ipv6 value to Connection instances

  • Added LINES attribute to Attr

  • Added pids_by_user()

  • Added address_to_int()

  • Added encoding()

  • Added datetime_to_unix()

  • Interpreter

  • Added a ‘–tor [path]’ argument to specify the tor binary to run.

  • Website

  • Version 1.5.3 (December 5th, 2016) - including tests and site in the release tarball

  • Version 1.5.4 (January 4th, 2017) - drop validation of the order of fields in the tor consensus (ticket 21059)

Version 1.4 (May 13th, 2015)

Stem’s 1.4 release brings with it new hidden service capabilities. Most notably, ephemeral hidden services and the ability to read hidden service descriptors. This release also changes descriptor validation to now be opt-in rather than opt-out. When unvalidated content is lazy-loaded, greatly improving our performance.

And last, Stem also now runs directly under both python2 and python3 without a 2to3 conversion (ticket 14075)!

  • Controller

  • Added Controller methods for a new style of hidden services that don’t touch disk: list_ephemeral_hidden_services(), create_ephemeral_hidden_service(), and remove_ephemeral_hidden_service() (spec)

  • Added get_hidden_service_descriptor() and support for HS_DESC_CONTENT events (ticket 14847, spec)

  • launch_tor_with_config() avoids writing a temporary torrc to disk if able (ticket 13865)

  • CircuitEvent support for the new SOCKS_USERNAME and SOCKS_PASSWORD arguments (ticket 14555, spec)

  • The ‘strict’ argument of can_exit_to() didn’t behave as documented (ticket 14314)

  • Threads spawned for status change listeners were never joined on, potentially causing noise during interpreter shutdown

  • Added support for specifying the authentication type and client names in create_hidden_service() (ticket 14320)

  • Descriptors

  • Lazy-loading descriptors, improving performance by 25-70% depending on what type it is (ticket 14011)

  • Added support for hidden service descriptors (ticket 15004)

  • When reading sanitised bridge descriptors (server or extrainfo), parse_file() treated the whole file as a single descriptor

  • The DirectoryAuthority ‘fingerprint’ attribute was actually its ‘v3ident’

  • Added consensus’ new package attribute (spec)

  • Added extra info’ new hs_stats_end, hs_rend_cells, hs_rend_cells_attr, hs_dir_onions_seen, and hs_dir_onions_seen_attr attributes (spec)

  • Updating Faravahar’s address (ticket 14487)

  • Utilities

  • Windows support for connection resolution (ticket 14844)

  • stem.util.connection.port_usage() always returned None (ticket 14046)

  • stylistic_issues() and pyflakes_issues() now provide namedtuples that also includes the line

  • Added stem.util.system.tail()

  • Proc connection resolution could fail on especially busy systems (ticket 14048)

  • Website

  • Version 1.4.1 (May 18th, 2015) - fixed issue where descriptors couldn’t be unpickled (ticket 16054) and a parsing issue for router status entry bandwidth lines (ticket 16048)

Version 1.3 (December 22nd, 2014)

With Stem’s 1.3 release it’s now much easier to work with hidden services, 40% faster to read decriptors, and includes a myriad of other improvements. For a nice description of the changes this brings see Nathan Willis’ LWN article.

  • Controller

  • Added Controller methods to more easily work with hidden service configurations: get_hidden_service_conf(), set_hidden_service_conf(), create_hidden_service(), and remove_hidden_service() (ticket 12533)

  • Added get_accounting_stats() to the Controller

  • Added get_effective_rate() to the Controller

  • Added connection_time() to the BaseController

  • Changed get_microdescriptor(), get_server_descriptor(), and get_network_status() to get our own descriptor if no fingerprint or nickname is provided.

  • Added ExitPolicy methods for more easily handling ‘private’ policies (the default prefix) and the defaultly appended suffix. This includes has_private(), strip_private(), has_default(), and strip_default() ExitPolicy methods in addition to is_private() and is_default() for the ExitPolicyRule. (ticket 10107)

  • Added the reason attribute to HSDescEvent (spec)

  • launch_tor_with_config() could cause a “Too many open files” OSError if called too many times (ticket 13141)

  • The get_exit_policy() method errored if tor couldn’t determine our external address

  • The Controller’s methods for retrieving descriptors could raise unexpected ValueErrors if tor didn’t have any descriptors available

  • Throwing a new DescriptorUnavailable exception type when the Controller can’t provide the descriptor for a relay (ticket 13879)

  • Descriptors

  • Utilities

  • Added support for directories to stem.util.conf.Config.load()

  • Changed stem.util.conf.uses_settings() to only provide a ‘config’ keyword arument if the decorated function would accept it

  • Added stem.util.str_tools.crop()

  • Added stem.util.proc.file_descriptors_used()

  • Dropped the ‘get_*’ prefix from most function names. Old names will still work, but are a deprecated alias.

  • Interpreter

  • The /info command errored for relays without contact information

  • Website

Version 1.2 (June 1st, 2014)

Stem release 1.2 added our interactive Tor interpreter among numerous other improvements and fixes.

  • Controller

  • New, better connect() function that deprecates connect_port() and connect_socket_file()

  • Added is_newnym_available() and get_newnym_wait() methods to the Controller

  • Added get_ports() and get_listeners() methods to the Controller

  • Added drop_guards() (ticket 10032, spec)

  • Added the id attribute to ORConnEvent (spec)

  • Added support for CONN_BW events (spec)

  • Added support for CIRC_BW events (spec)

  • Added support for CELL_STATS events (spec)

  • Added support for TB_EMPTY events (spec)

  • Added support for HS_DESC events (ticket 10807, spec)

  • Changed get_network_status() and get_network_statuses() to provide RouterStatusEntryMicroV3 if Tor is using microdescriptors (ticket 7646)

  • The connect_port() and connect_socket_file() didn’t properly mark the Controller it returned as being authenticated, causing event listening among other things to fail

  • The add_event_listener() method couldn’t accept event types that Stem didn’t already recognize

  • The ExitPolicy class couldn’t be pickled

  • Tor instances spawned with launch_tor() and launch_tor_with_config() could hang due to unread stdout content, we now close stdout and stderr once tor finishes bootstrapping (ticket 9862)

  • Descriptors

  • Added tarfile support to parse_file() (ticket 10977)

  • Added microdescriptor’s new identifier and identifier_type attributes (spec)

  • Utilities

  • Interpreter

  • Website

  • Added a section with example scripts.

  • Made FAQ and other sections quite a bit more succinct.

  • Version 1.2.2 (June 7th, 2014) - fixed an issue where the stem.util.conf module would fail under Python 2.6 with an AttributeError (ticket 12223)

  • Version 1.2.1 (June 3rd, 2014) - fixed an issue where descriptor parsersing would fail under Python 3.x with a TypeError (ticket 12185)

Version 1.1 (October 14th, 2013)

Stem release 1.1 introduced remote descriptor fetching, connection resolution and a myriad of smaller improvements and fixes.

  • Controller

  • get_network_status() and get_network_statuses() now provide v3 rather than v2 directory information (ticket 7953, spec)

  • AddrMapEvent support for the new CACHED argument (ticket 8596, spec)

  • attach_stream() could encounter an undocumented 555 response (ticket 8701, spec)

  • RelayDescriptor digest validation was broken when dealing with non-unicode content with Python 3 (ticket 8755)

  • The Controller use of cached content wasn’t thread safe (ticket 8607)

  • Added get_user() method to the Controller

  • Added get_pid() method to the Controller

  • StreamEvent didn’t recognize IPv6 addresses (ticket 9181)

  • get_conf() mistakenly cached hidden service related options (ticket 9792)

  • Added support for TRANSPORT_LAUNCHED events (spec)

  • Descriptors

  • Utilities

  • Connection resolution via the get_connections() function (ticket 7910)

  • set_process_name() inserted spaces between characters (ticket 8631)

  • pid_by_name() can now pull for all processes with a given name

  • call() ignored the subprocess’ exit status

  • Added stem.util.system.name_by_pid()

  • Added stem.util.system.user()

  • Added stem.util.system.start_time()

  • Added stem.util.system.bsd_jail_path()

  • Added stem.util.system.is_tarfile()

  • Added stem.util.connection.is_private_address()

  • Website

  • Overhaul of Stem’s download page. This included several improvements, most notably the addition of PyPI, Ubuntu, Fedora, Slackware, and FreeBSD.

  • Replaced default sphinx header with a navbar menu.

  • Added this change log.

  • Added the FAQ page.

  • Settled on a logo for Stem.

  • Expanded the client usage tutorial to cover SocksiPy and include an example for polling Twitter.

  • Subtler buttons for the frontpage (before and after).

  • Version 1.1.1 (November 9th, 2013) - fixed an issue where imports of stem.util.system would fail with an ImportError for pwd under Windows (ticket 10072)

Version 1.0 (March 26th, 2013)

This was the initial release of Stem.

  • Version 1.0.1 (March 27th, 2013) - fixed an issue where installing with Python 3.x (python3 setup.py install) resulted in a stacktrace