# api-simple-ipc The Simple-IPC API is a collection of `ipc_` prefixed library routines and a basic communication protocol that allows an IPC-client process to send an application-specific IPC-request message to an IPC-server process and receive an application-specific IPC-response message. Communication occurs over a named pipe on Windows and a Unix domain socket on other platforms. IPC-clients and IPC-servers rendezvous at a previously agreed-to application-specific pathname (which is outside the scope of this design) that is local to the computer system. The IPC-server routines within the server application process create a thread pool to listen for connections and receive request messages from multiple concurrent IPC-clients. When received, these messages are dispatched up to the server application callbacks for handling. IPC-server routines then incrementally relay responses back to the IPC-client. The IPC-client routines within a client application process connect to the IPC-server and send a request message and wait for a response. When received, the response is returned back to the caller. For example, the `fsmonitor--daemon` feature will be built as a server application on top of the IPC-server library routines. It will have threads watching for file system events and a thread pool waiting for client connections. Clients, such as `git status`, will request a list of file system events since a point in time and the server will respond with a list of changed files and directories. The formats of the request and response are application-specific; the IPC-client and IPC-server routines treat them as opaque byte streams. ## Comparison with sub-process model The Simple-IPC mechanism differs from the existing `sub-process.c` model (Documentation/technical/long-running-process-protocol.adoc) and used by applications like Git-LFS. In the LFS-style sub-process model, the helper is started by the foreground process, communication happens via a pair of file descriptors bound to the stdin/stdout of the sub-process, the sub-process only serves the current foreground process, and the sub-process exits when the foreground process terminates. In the Simple-IPC model the server is a very long-running service. It can service many clients at the same time and has a private socket or named pipe connection to each active client. It might be started (on-demand) by the current client process or it might have been started by a previous client or by the OS at boot time. The server process is not associated with a terminal and it persists after clients terminate. Clients do not have access to the stdin/stdout of the server process and therefore must communicate over sockets or named pipes. ## Server startup and shutdown How an application server based upon IPC-server is started is also outside the scope of the Simple-IPC design and is a property of the application using it. For example, the server might be started or restarted during routine maintenance operations, or it might be started as a system service during the system boot-up sequence, or it might be started on-demand by a foreground Git command when needed. Similarly, server shutdown is a property of the application using the simple- ipc routines. For example, the server might decide to shutdown when idle or only upon explicit request. ## Simple-ipc protocol The Simple-IPC protocol consists of a single request message from the client and an optional response message from the server. Both the client and server messages are unlimited in length and are terminated with a flush packet. The pkt-line routines ([gitprotocol-common[5]](gitprotocol-common)) are used to simplify buffer management during message generation, transmission, and reception. A flush packet is used to mark the end of the message. This allows the sender to incrementally generate and transmit the message. It allows the receiver to incrementally receive the message in chunks and to know when they have received the entire message. The actual byte format of the client request and server response messages are application specific. The IPC layer transmits and receives them as opaque byte buffers without any concern for the content within. It is the job of the calling application layer to understand the contents of the request and response messages. ## Summary Conceptually, the Simple-IPC protocol is similar to an HTTP REST request. Clients connect, make an application-specific and stateless request, receive an application-specific response, and disconnect. It is a one round trip facility for querying the server. The Simple-IPC routines hide the socket, named pipe, and thread pool details and allow the application layer to focus on the task at hand. # api-trace2 The Trace2 API can be used to print debug, performance, and telemetry information to stderr or a file. The Trace2 feature is inactive unless explicitly enabled by enabling one or more Trace2 Targets. The Trace2 API is intended to replace the existing (Trace1) `printf()`-style tracing provided by the existing `GIT_TRACE` and `GIT_TRACE_PERFORMANCE` facilities. During initial implementation, Trace2 and Trace1 may operate in parallel. The Trace2 API defines a set of high-level messages with known fields, such as (`start`: `argv`) and (`exit`: {`exit-code`, `elapsed-time`}). Trace2 instrumentation throughout the Git code base sends Trace2 messages to the enabled Trace2 Targets. Targets transform these messages content into purpose-specific formats and write events to their data streams. In this manner, the Trace2 API can drive many different types of analysis. Targets are defined using a VTable allowing easy extension to other formats in the future. This might be used to define a binary format, for example. Trace2 is controlled using `trace2.*` config values in the system and global config files and `GIT_TRACE2*` environment variables. Trace2 does not read from repo local or worktree config files, nor does it respect `-c` command line config settings. ## Trace2 targets Trace2 defines the following set of Trace2 Targets. Format details are given in a later section. ### The Normal Format Target The normal format target is a traditional `printf()` format and similar to the `GIT_TRACE` format. This format is enabled with the `GIT_TRACE2` environment variable or the `trace2.normalTarget` system or global config setting. For example $ export GIT_TRACE2=~/log.normal $ git version git version 2.20.1.155.g426c96fcdb or $ git config --global trace2.normalTarget ~/log.normal $ git version git version 2.20.1.155.g426c96fcdb yields $ cat ~/log.normal 12:28:42.620009 common-main.c:38 version 2.20.1.155.g426c96fcdb 12:28:42.620989 common-main.c:39 start git version 12:28:42.621101 git.c:432 cmd_name version (version) 12:28:42.621215 git.c:662 exit elapsed:0.001227 code:0 12:28:42.621250 trace2/tr2_tgt_normal.c:124 atexit elapsed:0.001265 code:0 ### The Performance Format Target The performance format target (PERF) is a column-based format to replace `GIT_TRACE_PERFORMANCE` and is suitable for development and testing, possibly to complement tools like `gprof`. This format is enabled with the `GIT_TRACE2_PERF` environment variable or the `trace2.perfTarget` system or global config setting. For example $ export GIT_TRACE2_PERF=~/log.perf $ git version git version 2.20.1.155.g426c96fcdb or $ git config --global trace2.perfTarget ~/log.perf $ git version git version 2.20.1.155.g426c96fcdb yields $ cat ~/log.perf 12:28:42.620675 common-main.c:38 | d0 | main | version | | | | | 2.20.1.155.g426c96fcdb 12:28:42.621001 common-main.c:39 | d0 | main | start | | 0.001173 | | | git version 12:28:42.621111 git.c:432 | d0 | main | cmd_name | | | | | version (version) 12:28:42.621225 git.c:662 | d0 | main | exit | | 0.001227 | | | code:0 12:28:42.621259 trace2/tr2_tgt_perf.c:211 | d0 | main | atexit | | 0.001265 | | | code:0 ### The Event Format Target The event format target is a JSON-based format of event data suitable for telemetry analysis. This format is enabled with the `GIT_TRACE2_EVENT` environment variable or the `trace2.eventTarget` system or global config setting. For example $ export GIT_TRACE2_EVENT=~/log.event $ git version git version 2.20.1.155.g426c96fcdb or $ git config --global trace2.eventTarget ~/log.event $ git version git version 2.20.1.155.g426c96fcdb yields $ cat ~/log.event {"event":"version","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.620713Z","file":"common-main.c","line":38,"evt":"4","exe":"2.20.1.155.g426c96fcdb"} {"event":"start","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621027Z","file":"common-main.c","line":39,"t_abs":0.001173,"argv":["git","version"]} {"event":"cmd_name","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621122Z","file":"git.c","line":432,"name":"version","hierarchy":"version"} {"event":"exit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621236Z","file":"git.c","line":662,"t_abs":0.001227,"code":0} {"event":"atexit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621268Z","file":"trace2/tr2_tgt_event.c","line":163,"t_abs":0.001265,"code":0} ### Enabling a Target To enable a target, set the corresponding environment variable or system or global config value to one of the following: * `0` or `false` \- Disables the target. * `1` or `true` \- Writes to `STDERR`. * `[2-9]` \- Writes to the already opened file descriptor. * `` \- Writes to the file in append mode. If the target already exists and is a directory, the traces will be written to files (one per process) underneath the given directory. * `af_unix:[:]` \- Write to a Unix DomainSocket (on platforms that support them). Socket type can be either `stream` or `dgram`; if omitted Git will try both. When trace files are written to a target directory, they will be named according to the last component of the SID (optionally followed by a counter to avoid filename collisions). ## Trace2 api The Trace2 public API is defined and documented in `trace2.h`; refer to it for more information. All public functions and macros are prefixed with `trace2_` and are implemented in `trace2.c`. There are no public Trace2 data structures. The Trace2 code also defines a set of private functions and data types in the `trace2/` directory. These symbols are prefixed with `tr2_` and should only be used by functions in `trace2.c` (or other private source files in `trace2/`). ### Conventions for Public Functions and Macros Some functions have a `_fl()` suffix to indicate that they take `file` and `line-number` arguments. Some functions have a `_va_fl()` suffix to indicate that they also take a `va_list` argument. Some functions have a `_printf_fl()` suffix to indicate that they also take a `printf()` style format with a variable number of arguments. CPP wrapper macros are defined to hide most of these details. ## Trace2 target formats ### NORMAL Format Events are written as lines of the form: [