Stem Docs

CollecTor

CollecTor

Descriptor archives are available from CollecTor. If you need Tor’s topology at a prior point in time this is the place to go!

With CollecTor you can either read descriptors directly…

import datetime
import stem.descriptor.collector

yesterday = datetime.datetime.utcnow() - datetime.timedelta(days = 1)

# provide yesterday's exits

exits = {}

for desc in stem.descriptor.collector.get_server_descriptors(start = yesterday):
  if desc.exit_policy.is_exiting_allowed():
    exits[desc.fingerprint] = desc

print('%i relays published an exiting policy today...\n' % len(exits))

for fingerprint, desc in exits.items():
  print('  %s (%s)' % (desc.nickname, fingerprint))

… or download the descriptors to disk and read them later.

import datetime
import stem.descriptor
import stem.descriptor.collector

yesterday = datetime.datetime.utcnow() - datetime.timedelta(days = 1)
cache_dir = '~/descriptor_cache/server_desc_today'

collector = stem.descriptor.collector.CollecTor()

for f in collector.files('server-descriptor', start = yesterday):
  f.download(cache_dir)

# then later...

for f in collector.files('server-descriptor', start = yesterday):
  for desc in f.read(cache_dir):
    if desc.exit_policy.is_exiting_allowed():
      print('  %s (%s)' % (desc.nickname, desc.fingerprint))

get_instance - Provides a singleton CollecTor used for...
  |- get_server_descriptors - published server descriptors
  |- get_extrainfo_descriptors - published extrainfo descriptors
  |- get_microdescriptors - published microdescriptors
  |- get_consensus - published router status entries
  |
  |- get_key_certificates - authority key certificates
  |- get_bandwidth_files - bandwidth authority heuristics
  +- get_exit_lists - TorDNSEL exit list

File - Individual file residing within CollecTor
  |- read - provides descriptors from this file
  +- download - download this file to disk

CollecTor - Downloader for descriptors from CollecTor
  |- get_server_descriptors - published server descriptors
  |- get_extrainfo_descriptors - published extrainfo descriptors
  |- get_microdescriptors - published microdescriptors
  |- get_consensus - published router status entries
  |
  |- get_key_certificates - authority key certificates
  |- get_bandwidth_files - bandwidth authority heuristics
  |- get_exit_lists - TorDNSEL exit list
  |
  |- index - metadata for content available from CollecTor
  +- files - files available from CollecTor

New in version 1.8.0.