Release Notes

v0.49.0rc1

Prelude

Several deprecations have been made to Context as we prepare to move active span management out of this class.

New Features

  • The futures integration is now enabled by default.

  • requests: add global config support. This enables the requests service name to be configured with ddtrace.config.requests['service'] or the DD_REQUESTS_SERVICE environment variable.

Upgrade Notes

  • Support for aiohttp previous to 2.0 has been removed.

  • Support for deprecated DD_PROFILING_API_URL environment variable has been removed. Use DD_SITE instead.

  • Support for deprecated DD_PROFILING_API_KEY environment variable has been removed. Use DD_API_KEY instead.

  • Profiling support for agentless mode must now be explicitly enabled.

  • The ddtrace pytest plugin can now label spans from test cases marked xfail with the tag “pytest.result”

    and the reason for being marked xfail under the tag “pytest.xfail.reason”.

  • requests: spans will no longer inherit the service name from the parent.

  • The return value of Span.pprint() has been changed to a single line in the tracer debug logs rather than the previous custom multiline format.

  • Spans are now processed per tracer instance. Formerly spans were stored per-Context which could be shared between tracer instances. Note that context management is not affected. Tracers will still share active spans.

  • Spans from asynchronous executions (asyncio, gevent, tornado) will now be processed and flushed together. Formerly the spans were handled per-task.

  • tracer.write() will no longer have filters applied to the spans passed to it.

  • The function ddtrace.utils.merge_dicts has been removed.

Deprecation Notes

  • Context.clone is deprecated. It will not be required in 0.50.

  • Context.add_span is deprecated and will be removed in 0.50.

  • Context.add_span is deprecated and will be removed in 0.50.

  • Context.close_span is deprecated and will be removed in 0.50.

  • Context.get_current_span is deprecated and will be removed in 0.50 please use Tracer.current_span instead.

  • Context.get_current_root_span is deprecated and will be removed in 0.50 please use Tracer.current_root_span instead.

  • Deprecate the configuration of the analytics through the generic dbapi2 configuration. This should now be configured via integration configurations, for example:

    # Before
    export DD_TRACE_DBAPI2_ANALYTICS_ENABLED=1
    
    # After
    export DD_TRACE_SQLITE3_ANALYTICS_ENABLED=1
    
  • ddtrace.compat has been deprecated and will be removed from the public API in ddtrace version 1.0.0.

  • Deprecate ddtrace.config.dbapi2 as default for TracedCursor and TracedConnection as well as DD_DBAPI2_TRACE_FETCH_METHODS. Use IntegrationConfig and DD_<INTEGRATION>_TRACE_FETCH_METHODS specific to each dbapi-compliant library. For example:

    # Before
    config.dbapi2.trace_fetch_methods = True
    
    # After
    config.psycopg2.trace_fetch_methods = True
    
  • The use of ddtrace.encoding has been deprecated and will be removed in version 1.0.0.

  • The ddtrace.http module has been deprecated and will be removed in version 1.0.0, with the ddtrace.http.headers module now merged into ddtrace.trace_utils.

  • The collect_metrics argument of the tracer.configure method has been deprecated. Runtime metrics should be enabled only via the DD_RUNTIME_METRICS_ENABLED environment variable.

Bug Fixes

  • Fix broken builds for Python 2.7 on windows where <stdint.h> was not available. This change also ensures we build and publish cp27-win wheels.

  • CGroup file parsing was fixed to correctly parse container ID with preceding characters.

  • grpc: handle None values for span tags.

  • grpc: handle no package in fully qualified method

  • grpc: Add done callback in streaming response to avoid unfinished spans if a StopIteration is never raised, as is found in the Google Cloud libraries.

  • grpc: handle IPv6 addresses and no port in target.

  • Fix DD_LOGS_INJECTION incompatibility when using a logging.StrFormatStyle (logging.Formatter(fmt, style="{")) log formatter.

  • Fixed a bug that prevented the right integration name to be used when trying to patch a module on import that is already loaded.

  • Fix urllib3 patching not properly activating the integration.

  • gRPC client spans are now marked as measured by default.

  • Fixes issue of unfinished spans when response is not a grpc.Future but has the same interface, as is the case with the base future class in google-api-core.

  • In certain circumstances, the profiles generated in a uWSGI application could have been empty. This is now fixed and the profiler records correctly the generated events.

  • The default agent timeout for profiling has been restored from 2 to 10 seconds to avoid too many profiles from being dropped.

  • Fix issue with missing traces when using pymemcache.client.hash.HashClient.

  • Added missing pymongo integration configuration, which allows overriding the service name for all the emitted spans.

Other Notes

  • Added environment variable DD_BOTTLE_DISTRIBUTED_TRACING to enable distributed tracing for bottle.

  • The attrs library has been unvendored and is now required as a normal Python dependency with a minimum version requirement of 19.2.0.

  • The six library has been removed from vendor and the system-wide version is being used. It requires version 1.12.0 or later.

  • Documentation on how to use Gunicorn with the gevent worker class has been added.

  • Added environment variable DD_FALCON_DISTRIBUTED_TRACING to enable distributed tracing for falcon.

  • When extracting context information from HTTP headers, a new context is created when the trace ID is either 0 or not available within the headers.

  • Added environment variable DD_PYLONS_DISTRIBUTED_TRACING to enable distributed tracing for pylons.

  • Update pymemcache test suite to test latest versions.

  • Added config.pyramid.distributed_tracing setting to integration config for pyramid.

  • Removed ddtrace.ext.AppTypes and its usages in the tracer library.

  • The ddtrace.payload submodule has been removed.

  • Added environment variable DD_TORNADO_DISTRIBUTED_TRACING to enable distributed tracing for tornado.

v0.48.0rc2

New Features

  • Adds support for the Datadog Lambda Extension. The tracer will send traces to the extension by default if it is present.

  • Add support for space-separated DD_TAGS.

  • urllib3: add urllib3 integration

v0.48.0rc1

New Features

  • dogpile.cache: is now automatically instrumented by default.

  • pylons: now supports all the standard http tagging including query string, custom error codes, and request/response headers.

  • The ddtrace pytest plugin can now call ddtrace.patch_all via the --ddtrace-patch-all option.

  • Span now accepts a on_finish argument used for specifying functions to call when a span finishes.

Upgrade Notes

  • The deprecated dogstatsd_host and dogstatsd_port arguments to tracer.configure() have been removed.

  • Support for gevent 1.0 has been removed and gevent >= 1.1 is required.

  • flask: deprecated configuration option extra_error_codes has been removed.

  • The deprecated pyddprofile wrapper has been removed. Use ddtrace-run with DD_PROFILING_ENABLED=1 set instead.

  • A ValueError will now be raised on tracer initialization if the Agent URL specified to the initializer or with the environment variable DD_TRACE_AGENT_URL is malformed.

  • uWSGI is no longer supported with ddtrace-run due to a limitation of how tracer initialization occurs. See the updated instructions for enabling tracing in the library uWSGI documentation.

Bug Fixes

  • The Records parameter to Firehose endpoint calls is being excluded from the tags to avoid generating traces with a large payload.

  • Tracer: fix configuring tracer with dogstatsd url.

  • elasticsearch: patch versioned elasticsearch modules (elasticsearch1, …, elasticsearch7).

  • botocore: Do not assume that ResponseMeta exists in the results.

  • django: handle erroneous middleware gracefully.

  • The tracer now captures the task ID from the cgroups file for Fargate >= 1.4.0 and reports it to the agent as the Datadog-Container-ID tag.

  • Fix a bug when tracing a mako DefTemplate or any Template that does not have a filename property.

  • flask: fix a bug when the query string would contain non-Unicode characters

  • Fix a formatting issue on error exporting profiles.

  • A workaround is provided for the problem with uWSGI worker processes failing to respawn. This can occur when using ddtrace-run for automatic instrumentation and configuration or manual instrumentation and configuration without the necessary uWSGI options. The problem is caused by how the tracer can end up starting threads in the master process before uWSGI forks to initialize the workers processes. To avoid this, we have provided updated instructions for enabling tracing in the library uWSGI documentation.

Other Notes

  • The logic behind the header extraction for distributed tracing has been improved.

  • The default connection timeout for the profiling agent has now been reduced from 10 to 2 seconds to match the tracer behavior.

  • The tracemalloc memory profiler, which was disabled by default, has been removed.

  • Query strings are stripped out from URLs by default when setting URL metadata on a span. This change affects all integrations that store HTTP metadata, like aiohttp, falcon, requests, urllib3.

v0.47.0rc1

New Features

  • cherrypy: introduce TraceMiddleware for the CherryPy web framework.

  • django: tag root spans as measured.

  • elasticsearch: add support for version 7.

  • fastapi: add integration.

  • Introduce support for the DD_SERVICE_MAPPING environment variable to allow remapping service names on emitted spans.

  • httplib: distributed tracing is now enabled by default.

  • The profiler now supports most operation mode from uWSGI without much configuration. It will automatically plug itself in post fork hooks when multiprocess mode is used.

  • wsgi: add tracing middleware.

Upgrade Notes

  • elasticsearch: removed get_traced_transport method and ddtrace.contrib.elasticsearch.transport module.

  • The profiler now automatically sets up uWSGI compatibility in auto mode or with profile_children=True. Make sure that you don’t have custom code instrumenting the profiler in those cases.

  • The Tracer class properties DEFAULT_HOSTNAME, DEFAULT_PORT, DEFAULT_DOGSTATSD_PORT, DEFAULT_DOGSTATSD_URL, DEFAULT_AGENT_URL have been removed.

Bug Fixes

  • Resolves an issue in Django tracing where, if query_string is not present in request.META, a KeyError is raised, causing the request to 500

  • Deprecate the DD_LOGGING_RATE_LIMIT variable in favor of the standard DD_TRACE_LOGGING_RATE for configuring the logging rate limit.

  • sampler: removed bug causing sample_rate of 0 to be reset to 1, and raise ValueError instead of logging.

  • starlette: unpatch calls correctly.

  • flask: fix memory leak of sampled out traces.

  • Fix CPU time and wall time profiling not ignoring the profiler tasks with gevent.

Other Notes

  • The default maximum CPU time used for the stack profiler (CPU time, wall time and exceptions profiling) has been decreased from 2% to 1%.

v0.46.0rc1

New Features

  • aiohttp: store request and response headers.

  • bottle: store request and response headers.

  • flask: store response headers.

  • molten: store request headers.

  • pyramid: store request and response headers.

  • flask: store request and response headers when auto-instrumented.

  • The ddtrace.profiling.auto module will warn users if gevent monkey patching is done after the profiler is auto-instrumented.

  • The Profiler object can now be passed tags with the tags keyword argument.

Upgrade Notes

  • The profiler will only load tags from the DD_TAGS environment variable once at start.

Deprecation Notes

Bug Fixes

  • dbapi: avoid type error with potential non-compliance in db libraries when setting tag for row count.

  • django: add legacy resource format of {handler}.

  • grpc: fix wrapper for streaming response to support libraries that call an internal method directly.

  • sanic: use path parameter names instead of parameter values for the resource.

  • The profiler won’t deadlock on fork when gevent monkey patch is enabled.

  • requests: fix TracedSession when patches are not applied.

v0.45.0rc1

Prelude

Build and deploy Python 3.9 wheels for releases

New Features

  • botocore: added distributed_tracing configuration setting which is enabled by default.

  • The ddtrace-run command now supports the following arguments:

    -h, –help -d, –debug enable debug mode (disabled by default) -i, –info print library info useful for debugging -p, –profiling enable profiling (disabled by default) -v, –version show program’s version number and exit

    It now also has friendlier error messages when used incorrectly.

  • Add functionality to call gevent.monkey.patch_all() with ddtrace-run by setting the environment variable DD_GEVENT_PATCH_ALL=true. This ensures that gevent patching is done as early as possible in the application.

  • botocore: inject distributed tracing data to ClientContext to trace lambda invocations.

  • botocore: inject tracing data to MessageAttributes.

  • The profiler now tracks the running gevent Greenlet and store it as part of the CPU and wall time profiling information.

  • The profiler is now able to upload profiles to the Datadog Agent by using a Unix Domain Socket.

  • It is now possible to pass a url parameter to the Profiler to specify the Datadog agent location.

  • The new memory profiler for Python is now enabled by default. This improves the profiler memory consumption and performance impact. It can still be disabled by setting DD_PROFILING_MEMALLOC=0 as an environment variable.

  • The profiler now uses the tracer configuration is no configuration is provided.

  • pytest integration. This enables the pytest runner to trace test executions.

Upgrade Notes

  • Context.get() has been removed and the functionality has been rolled into Context.close_span().

  • Tracer.record() has similarly been removed as it is no longer useful with Context.get() removed.

  • The deprecated compatibility module ddtrace.profile has been removed.

  • The profiler now uses the tracer configuration is no configuration is provided.

Deprecation Notes

  • The pyddprofile wrapper is deprecated. Use DD_PROFILING_ENABLED=true ddtrace-run instead.

  • The profiler does not catch uncaught exception anymore.

Bug Fixes

  • core: always reset the current_span in the context.

  • django: Http404 exceptions will no longer be flagged as errors

  • django: add safe guards for building http.url span tag.

  • aiobotocore: set span error for 5xx status codes.

  • elasticsearch: set span error for 5xx status codes.

  • django, DRF, ASGI: fix span type for web request spans.

  • Fixes span id tagging in lock profiling.

  • Fix UDS upload for profiling not using the correct path.

  • Fixed an issue in profiling exporting profiles twice when forking.

  • core: fix race condition in TracerTagCollector.

Other Notes

  • Start-up logs are now disabled by default. To enable start-up logs use DD_TRACE_STARTUP_LOGS=true or DD_TRACE_DEBUG=true.

v0.44.0

Prelude

Add support for Python 3.9

New Features

  • Store request headers in Flask integration.

  • pyodbc integration. This enables the pyodbc library to trace queries.

  • starlette integration resource aggregation This aggregates endpoints to the starlette application resource that was accessed. It occurs by default but it is configurable through config.starlette[“aggregate_resources”].

  • The profiler now captures the traces information with the lock profiling.

  • The Profiler instances now restart automatically in child process when the main program is forked. This only works for Python ≥ 3.7.

Bug Fixes

  • dbapi: add support for connection context manager usage

  • django: check view before instrumenting MRO.

  • core: use loose types when encoding.

  • Patch pynamodb on import to prevent patching conflicts with gevent.

  • tornado: handle when the current span is None in log_exception().

Prior Releases

Release notes prior to v0.44.0 can be found in CHANGELOG.md in the root of the repository.