Extrainfo Descriptor¶
Parsing for Tor extra-info descriptors. These are published by relays whenever their server descriptor is published and have a similar format. However, unlike server descriptors these don’t contain information that Tor clients require to function and as such aren’t fetched by default.
Defined in section 2.1.2 of the dir-spec, extra-info descriptors contain interesting but non-vital information such as usage statistics. Tor clients cannot request these documents for bridges.
Extra-info descriptors are available from a few sources…
- If you have ‘DownloadExtraInfo 1’ in your torrc…
- control port via ‘GETINFO extra-info/digest/*’ queries
- the ‘cached-extrainfo’ file in tor’s data directory
- Archived descriptors provided by CollecTor.
- Directory authorities and mirrors via their DirPort.
Module Overview:
ExtraInfoDescriptor - Tor extra-info descriptor.
|- RelayExtraInfoDescriptor - Extra-info descriptor for a relay.
|- BridgeExtraInfoDescriptor - Extra-info descriptor for a bridge.
|
+- digest - calculates the upper-case hex digest value for our content
-
stem.descriptor.extrainfo_descriptor.
DirResponse
(enum)¶ Enumeration for known statuses for ExtraInfoDescriptor’s dir_*_responses.
DirResponse Description OK network status requests that were answered NOT_ENOUGH_SIGS network status wasn’t signed by enough authorities UNAVAILABLE requested network status was unavailable NOT_FOUND requested network status was not found NOT_MODIFIED network status unmodified since If-Modified-Since time BUSY directory was busy
-
stem.descriptor.extrainfo_descriptor.
DirStat
(enum)¶ Enumeration for known stats for ExtraInfoDescriptor’s dir_*_direct_dl and dir_*_tunneled_dl.
DirStat Description COMPLETE requests that completed successfully TIMEOUT requests that didn’t complete within a ten minute timeout RUNNING requests still in process when measurement’s taken MIN smallest rate at which a descriptor was downloaded in B/s MAX largest rate at which a descriptor was downloaded in B/s D1-4 and D6-9 rate of the slowest x/10 download rates in B/s Q1 and Q3 rate of the slowest and fastest quarter download rates in B/s MD median download rate in B/s
-
class
stem.descriptor.extrainfo_descriptor.
ExtraInfoDescriptor
(raw_contents, validate=False)[source]¶ Bases:
stem.descriptor.Descriptor
Extra-info descriptor document.
Variables: - nickname (str) – * relay’s nickname
- fingerprint (str) – * identity key fingerprint
- published (datetime) – * time in UTC when this descriptor was made
- geoip_db_digest (str) – sha1 of the geoIP database file for IPv4 addresses
- geoip6_db_digest (str) – sha1 of the geoIP database file for IPv6 addresses
- transport (dict) – * mapping of transport methods to their (address, port, args) tuple, these usually appear on bridges in which case all of those are None
Bi-directional connection usage:
Variables: - conn_bi_direct_end (datetime) – end of the sampling interval
- conn_bi_direct_interval (int) – seconds per interval
- conn_bi_direct_below (int) – connections that read/wrote less than 20 KiB
- conn_bi_direct_read (int) – connections that read at least 10x more than wrote
- conn_bi_direct_write (int) – connections that wrote at least 10x more than read
- conn_bi_direct_both (int) – remaining connections
Bytes read/written for relayed traffic:
Variables: - read_history_end (datetime) – end of the sampling interval
- read_history_interval (int) – seconds per interval
- read_history_values (list) – bytes read during each interval
- write_history_end (datetime) – end of the sampling interval
- write_history_interval (int) – seconds per interval
- write_history_values (list) – bytes written during each interval
Cell relaying statistics:
Variables: - cell_stats_end (datetime) – end of the period when stats were gathered
- cell_stats_interval (int) – length in seconds of the interval
- cell_processed_cells (list) – measurement of processed cells per circuit
- cell_queued_cells (list) – measurement of queued cells per circuit
- cell_time_in_queue (list) – mean enqueued time in milliseconds for cells
- cell_circuits_per_decile (int) – mean number of circuits in a decile
Directory Mirror Attributes:
Variables: - dir_stats_end (datetime) – end of the period when stats were gathered
- dir_stats_interval (int) – length in seconds of the interval
- dir_v2_ips (dict) – mapping of locales to rounded count of requester ips
- dir_v3_ips (dict) – mapping of locales to rounded count of requester ips
- dir_v2_share (float) – percent of total directory traffic it expects to serve
- dir_v3_share (float) – percent of total directory traffic it expects to serve
- dir_v2_requests (dict) – mapping of locales to rounded count of requests
- dir_v3_requests (dict) – mapping of locales to rounded count of requests
- dir_v2_responses (dict) – mapping of
DirResponse
to their rounded count - dir_v3_responses (dict) – mapping of
DirResponse
to their rounded count - dir_v2_responses_unknown (dict) – mapping of unrecognized statuses to their count
- dir_v3_responses_unknown (dict) – mapping of unrecognized statuses to their count
- dir_v2_direct_dl (dict) – mapping of
DirStat
to measurement over DirPort - dir_v3_direct_dl (dict) – mapping of
DirStat
to measurement over DirPort - dir_v2_direct_dl_unknown (dict) – mapping of unrecognized stats to their measurement
- dir_v3_direct_dl_unknown (dict) – mapping of unrecognized stats to their measurement
- dir_v2_tunneled_dl (dict) – mapping of
DirStat
to measurement over ORPort - dir_v3_tunneled_dl (dict) – mapping of
DirStat
to measurement over ORPort - dir_v2_tunneled_dl_unknown (dict) – mapping of unrecognized stats to their measurement
- dir_v3_tunneled_dl_unknown (dict) – mapping of unrecognized stats to their measurement
Bytes read/written for directory mirroring:
Variables: - dir_read_history_end (datetime) – end of the sampling interval
- dir_read_history_interval (int) – seconds per interval
- dir_read_history_values (list) – bytes read during each interval
- dir_write_history_end (datetime) – end of the sampling interval
- dir_write_history_interval (int) – seconds per interval
- dir_write_history_values (list) – bytes read during each interval
Guard Attributes:
Variables: - entry_stats_end (datetime) – end of the period when stats were gathered
- entry_stats_interval (int) – length in seconds of the interval
- entry_ips (dict) – mapping of locales to rounded count of unique user ips
Exit Attributes:
Variables: - exit_stats_end (datetime) – end of the period when stats were gathered
- exit_stats_interval (int) – length in seconds of the interval
- exit_kibibytes_written (dict) – traffic per port (keys are ints or ‘other’)
- exit_kibibytes_read (dict) – traffic per port (keys are ints or ‘other’)
- exit_streams_opened (dict) – streams per port (keys are ints or ‘other’)
Hidden Service Attributes:
Variables: - hs_stats_end (datetime) – end of the sampling interval
- hs_rend_cells (int) – rounded count of the RENDEZVOUS1 cells seen
- hs_rend_cells_attr (int) – * attributes provided for the hs_rend_cells
- hs_dir_onions_seen (int) – rounded count of the identities seen
- hs_dir_onions_seen_attr (int) – * attributes provided for the hs_dir_onions_seen
Padding Count Attributes:
Variables: - padding_counts (dict) – * padding parameters
- padding_counts_end (datetime) – end of the period when padding data is being collected
- padding_counts_interval (int) – length in seconds of the interval
Bridge Attributes:
Variables: - bridge_stats_end (datetime) – end of the period when stats were gathered
- bridge_stats_interval (int) – length in seconds of the interval
- bridge_ips (dict) – mapping of locales to rounded count of unique user ips
- geoip_start_time (datetime) – replaced by bridge_stats_end (deprecated)
- geoip_client_origins (dict) – replaced by bridge_ips (deprecated)
- ip_versions (dict) – mapping of ip protocols to a rounded count for the number of users
- ip_versions – mapping of ip transports to a count for the number of users
* attribute is either required when we’re parsed with validation or has a default value, others are left as None if undefined
Changed in version 1.4.0: Added the hs_stats_end, hs_rend_cells, hs_rend_cells_attr, hs_dir_onions_seen, and hs_dir_onions_seen_attr attributes.
Changed in version 1.6.0: Added the padding_counts, padding_counts_end, and padding_counts_interval attributes.
-
digest
(hash_type='SHA1', encoding='HEX')[source]¶ Digest of this descriptor’s content. These are referenced by…
- Server Descriptors
- Referer:
ServerDescriptor
extra_info_digest attribute - Format: SHA1/HEX
- Referer:
- Server Descriptors
- Referer:
ServerDescriptor
extra_info_sha256_digest attribute - Format: SHA256/BASE64
- Referer:
Changed in version 1.8.0: Added the hash_type and encoding arguments.
Parameters: - hash_type (stem.descriptor.DigestHash) – digest hashing algorithm
- encoding (stem.descriptor.DigestEncoding) – digest encoding
Returns: hashlib.HASH or str based on our encoding argument
- Server Descriptors
-
class
stem.descriptor.extrainfo_descriptor.
RelayExtraInfoDescriptor
(raw_contents, validate=False)[source]¶ Bases:
stem.descriptor.extrainfo_descriptor.ExtraInfoDescriptor
Relay extra-info descriptor, constructed from data such as that provided by ‘GETINFO extra-info/digest/*’, cached descriptors, and metrics (specification).
Variables: - str (ed25519_signature) – base64 encoded ed25519 certificate
- str – signature of this document using ed25519
- signature (str) – * signature for this extrainfo descriptor
* attribute is required when we’re parsed with validation
Changed in version 1.5.0: Added the ed25519_certificate and ed25519_signature attributes.
-
TYPE_ANNOTATION_NAME
= 'extra-info'¶
-
classmethod
content
(attr=None, exclude=(), sign=False, signing_key=None)[source]¶ Creates descriptor content with the given attributes. Mandatory fields are filled with dummy information unless data is supplied. This doesn’t yet create a valid signature.
New in version 1.6.0.
Parameters: - attr (dict) – keyword/value mappings to be included in the descriptor
- exclude (list) – mandatory keywords to exclude from the descriptor, this results in an invalid descriptor
- sign (bool) – includes cryptographic signatures and digests if True
Returns: str with the content of a descriptor
Raises: - ImportError if cryptography is unavailable and sign is True
- NotImplementedError if not implemented for this descriptor type
-
classmethod
create
(attr=None, exclude=(), validate=True, sign=False, signing_key=None)[source]¶ Creates a descriptor with the given attributes. Mandatory fields are filled with dummy information unless data is supplied. This doesn’t yet create a valid signature.
New in version 1.6.0.
Parameters: - attr (dict) – keyword/value mappings to be included in the descriptor
- exclude (list) – mandatory keywords to exclude from the descriptor, this results in an invalid descriptor
- validate (bool) – checks the validity of the descriptor’s content if True, skips these checks otherwise
- sign (bool) – includes cryptographic signatures and digests if True
Returns: Descriptor
subclassRaises: - ValueError if the contents is malformed and validate is True
- ImportError if cryptography is unavailable and sign is True
- NotImplementedError if not implemented for this descriptor type
-
class
stem.descriptor.extrainfo_descriptor.
BridgeExtraInfoDescriptor
(raw_contents, validate=False)[source]¶ Bases:
stem.descriptor.extrainfo_descriptor.ExtraInfoDescriptor
Bridge extra-info descriptor (bridge descriptor specification)
Variables: - ed25519_certificate_hash (str) – sha256 hash of the original identity-ed25519
- router_digest_sha256 (str) – sha256 digest of this document
Changed in version 1.5.0: Added the ed25519_certificate_hash and router_digest_sha256 attributes.
-
TYPE_ANNOTATION_NAME
= 'bridge-extra-info'¶
-
classmethod
content
(attr=None, exclude=(), sign=False)[source]¶ Creates descriptor content with the given attributes. Mandatory fields are filled with dummy information unless data is supplied. This doesn’t yet create a valid signature.
New in version 1.6.0.
Parameters: - attr (dict) – keyword/value mappings to be included in the descriptor
- exclude (list) – mandatory keywords to exclude from the descriptor, this results in an invalid descriptor
- sign (bool) – includes cryptographic signatures and digests if True
Returns: str with the content of a descriptor
Raises: - ImportError if cryptography is unavailable and sign is True
- NotImplementedError if not implemented for this descriptor type
-
digest
(hash_type='SHA1', encoding='HEX')[source]¶ Digest of this descriptor’s content. These are referenced by…
- Server Descriptors
- Referer:
ServerDescriptor
extra_info_digest attribute - Format: SHA1/HEX
- Referer:
- Server Descriptors
- Referer:
ServerDescriptor
extra_info_sha256_digest attribute - Format: SHA256/BASE64
- Referer:
Changed in version 1.8.0: Added the hash_type and encoding arguments.
Parameters: - hash_type (stem.descriptor.DigestHash) – digest hashing algorithm
- encoding (stem.descriptor.DigestEncoding) – digest encoding
Returns: hashlib.HASH or str based on our encoding argument
- Server Descriptors