Spring Boot - How to log all requests and responses with exceptions in single place

Question

I m working on rest api with spring boot  I need to log all requests with input params  with methods  eg  GET  POST  etc   request path  query string  corresponding class method of this request  also response of this action  both success and errors  For example  successful request  http   example com api users 1  Log should look something like this       HttpStatus  200     path   quot api users 1 quot      method   quot GET quot      clientIp   quot 0 0 0 0 quot      accessToken   quot XHGu6as5dajshdgau6i6asdjhgjhg quot      method   quot UsersController getUser quot      arguments         id  1           response          user            id  1          username   quot user123 quot           email   quot user123 example com quot                      exceptions               Or request with error  http   example com api users 9999  Log should be something like this       HttpStatus  404     errorCode  101                      path   quot api users 9999 quot      method   quot GET quot      clientIp   quot 0 0 0 0 quot      accessToken   quot XHGu6as5dajshdgau6i6asdjhgjhg quot      method   quot UsersController getUser quot      arguments         id  9999           returns                         exceptions                  exception   quot UserNotFoundException quot          message   quot User with id 9999 not found quot          exceptionId   quot adhaskldjaso98d7324kjh989 quot          stacktrace                                         I want Request Response to be a single entity  with custom information related to this entity  both in successful and error cases  What is best practice in spring to achieve this  may be with filters  if yes  can you provide concrete example  I ve played with  ControllerAdvice and  ExceptionHandler  but as I mentioned  I need to handle all success and error requests in single place  and single log

User · Answer

The Logbook library is specifically made for logging HTTP requests and responses. It supports Spring Boot using a special starter library.

To enable logging in Spring Boot all you need to do is adding the library to your project's dependencies. For example assuming you are using Maven:

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-spring-boot-starter</artifactId>
    <version>1.5.0</version>
</dependency>

By default the logging output looks like this:

{
  "origin" : "local",
  "correlation" : "52e19498-890c-4f75-a06c-06ddcf20836e",
  "status" : 200,
  "headers" : {
    "X-Application-Context" : [
      "application:8088"
    ],
    "Content-Type" : [
      "application/json;charset=UTF-8"
    ],
    "Transfer-Encoding" : [
      "chunked"
    ],
    "Date" : [
      "Sun, 24 Dec 2017 13:10:45 GMT"
    ]
  },
  "body" : {
    "thekey" : "some_example"
  },
  "duration" : 105,
  "protocol" : "HTTP/1.1",
  "type" : "response"
}

It does however not output the class name that is handling the request. The library does have some interfaces for writing custom loggers.

Notes

In the meantime the library has significantly evolved, current version is 2.4.1, see https://github.com/zalando/logbook/releases. E.g. the default ouput format has changed, and can be configured, filtered, etc.

Do NOT forget to set the log level to TRACE, else you won't see anything:

logging:
  level:
    org.zalando.logbook: TRACE

User · Answer

Here is how I do it in spring data rest by using org springframework web util ContentCachingRequestWrapper  and org springframework web util ContentCachingResponseWrapper         Doogies very cool HTTP request logging       There is also   link org springframework web filter CommonsRequestLoggingFilter   but it cannot log request method    And it cannot easily be extended        https   mdeinum wordpress com 2015 07 01 spring-framework-hidden-gems     http   stackoverflow com questions 8933054 how-to-read-and-copy-the-http-servlet-response-output-stream-content-for-logging     public class DoogiesRequestLogger extends OncePerRequestFilter      private boolean includeResponsePayload   true    private int maxPayloadLength   1000     private String getContentAsString byte   buf  int maxLength  String charsetName        if  buf    null    buf length    0  return         int length   Math min buf length  this maxPayloadLength       try         return new String buf  0  length  charsetName         catch  UnsupportedEncodingException ex          return  Unsupported Encoding                         Log each request and respponse with full Request URI  content payload and duration of the request in ms        param request the request       param response the response       param filterChain chain of filters       throws ServletException       throws IOException          Override   protected void doFilterInternal HttpServletRequest request  HttpServletResponse response  FilterChain filterChain  throws ServletException  IOException        long startTime   System currentTimeMillis        StringBuffer reqInfo   new StringBuffer         append            append startTime   10000      request ID       append             append request getMethod          append            append request getRequestURL          String queryString   request getQueryString        if  queryString    null          reqInfo append      append queryString              if  request getAuthType      null          reqInfo append    authType             append request getAuthType               if  request getUserPrincipal      null          reqInfo append    principalName             append request getUserPrincipal   getName                this logger debug    gt      reqInfo                     Log request and response payload   body                   We CANNOT simply read the request payload here  because then the InputStream would be consumed and cannot be read again by the actual processing server            String reqBody   DoogiesUtil  stream2String request getInputStream          THIS WOULD NOT WORK         So we need to apply some stronger magic here  -      ContentCachingRequestWrapper wrappedRequest   new ContentCachingRequestWrapper request       ContentCachingResponseWrapper wrappedResponse   new ContentCachingResponseWrapper response        filterChain doFilter wrappedRequest  wrappedResponse                   This performs the actual request      long duration   System currentTimeMillis   - startTime          I can only log the request s body AFTER the request has been made and ContentCachingRequestWrapper did its work      String requestBody   this getContentAsString wrappedRequest getContentAsByteArray    this maxPayloadLength  request getCharacterEncoding         if  requestBody length    gt  0          this logger debug     Request body  n   requestBody              this logger debug   lt       reqInfo      returned status     response getStatus       in   duration    ms        if  includeResponsePayload          byte   buf   wrappedResponse getContentAsByteArray          this logger debug     Response body  n  getContentAsString buf  this maxPayloadLength  response getCharacterEncoding                 wrappedResponse copyBodyToResponse        IMPORTANT  copy content of response back into original response

User · Answer

This code works for me in a Spring Boot application - just register it as a filter      import java io BufferedReader      import java io ByteArrayInputStream      import java io ByteArrayOutputStream      import java io IOException      import java io InputStream      import java io InputStreamReader      import java io OutputStream      import java io PrintWriter      import java util Collection      import java util Enumeration      import java util HashMap      import java util Locale      import java util Map      import javax servlet        import javax servlet http Cookie      import javax servlet http HttpServletRequest      import javax servlet http HttpServletRequestWrapper      import javax servlet http HttpServletResponse      import org apache commons io output TeeOutputStream      import org slf4j Logger      import org slf4j LoggerFactory      import org springframework stereotype Component        Component     public class HttpLoggingFilter implements Filter            private static final Logger log   LoggerFactory getLogger HttpLoggingFilter class             Override         public void init FilterConfig filterConfig  throws ServletException                       Override         public void doFilter ServletRequest request  ServletResponse response                               FilterChain chain  throws IOException  ServletException               try                   HttpServletRequest httpServletRequest    HttpServletRequest  request                  HttpServletResponse httpServletResponse    HttpServletResponse  response                   Map lt String  String gt  requestMap   this                          getTypesafeRequestMap httpServletRequest                   BufferedRequestWrapper bufferedRequest   new BufferedRequestWrapper                          httpServletRequest                   BufferedResponseWrapper bufferedResponse   new BufferedResponseWrapper                          httpServletResponse                    final StringBuilder logMessage   new StringBuilder                           REST Request -    append   HTTP METHOD                             append httpServletRequest getMethod                             append     PATH INFO                             append httpServletRequest getServletPath                             append     REQUEST PARAMETERS    append requestMap                           append     REQUEST BODY                             append bufferedRequest getRequestBody                             append     REMOTE ADDRESS                             append httpServletRequest getRemoteAddr    append                        chain doFilter bufferedRequest  bufferedResponse                   logMessage append    RESPONSE                             append bufferedResponse getContent    append                       log debug logMessage toString                   catch  Throwable a                    log error a getMessage                                      private Map lt String  String gt  getTypesafeRequestMap HttpServletRequest request                Map lt String  String gt  typesafeRequestMap   new HashMap lt String  String gt                 Enumeration lt   gt  requestParamNames   request getParameterNames                while  requestParamNames hasMoreElements                      String requestParamName    String  requestParamNames nextElement                    String requestParamValue                  if  requestParamName equalsIgnoreCase  password                          requestParamValue                                 else                       requestParamValue   request getParameter requestParamName                                     typesafeRequestMap put requestParamName  requestParamValue                             return typesafeRequestMap                      Override         public void destroy                        private static final class BufferedRequestWrapper extends                 HttpServletRequestWrapper                private ByteArrayInputStream bais   null              private ByteArrayOutputStream baos   null              private BufferedServletInputStream bsis   null              private byte   buffer   null               public BufferedRequestWrapper HttpServletRequest req                      throws IOException                   super req                      Read InputStream and store its content in a buffer                  InputStream is   req getInputStream                    this baos   new ByteArrayOutputStream                    byte buf     new byte 1024                   int read                  while   read   is read buf    gt  0                        this baos write buf  0  read                                     this buffer   this baos toByteArray                                Override             public ServletInputStream getInputStream                     this bais   new ByteArrayInputStream this buffer                   this bsis   new BufferedServletInputStream this bais                   return this bsis                             String getRequestBody   throws IOException                   BufferedReader reader   new BufferedReader new InputStreamReader                          this getInputStream                      String line   null                  StringBuilder inputBuffer   new StringBuilder                    do                       line   reader readLine                        if  null    line                            inputBuffer append line trim                                             while  line    null                   reader close                    return inputBuffer toString   trim                                      private static final class BufferedServletInputStream extends                 ServletInputStream                private ByteArrayInputStream bais               public BufferedServletInputStream ByteArrayInputStream bais                    this bais   bais                              Override             public int available                     return this bais available                                Override             public int read                     return this bais read                                Override             public int read byte   buf  int off  int len                    return this bais read buf  off  len                               Override             public boolean isFinished                     return false                              Override             public boolean isReady                     return true                              Override             public void setReadListener ReadListener readListener                                      public class TeeServletOutputStream extends ServletOutputStream                private final TeeOutputStream targetStream               public TeeServletOutputStream OutputStream one  OutputStream two                    targetStream   new TeeOutputStream one  two                               Override             public void write int arg0  throws IOException                   this targetStream write arg0                              public void flush   throws IOException                   super flush                    this targetStream flush                               public void close   throws IOException                   super close                    this targetStream close                                Override             public boolean isReady                     return false                              Override             public void setWriteListener WriteListener writeListener                                      public class BufferedResponseWrapper implements HttpServletResponse                HttpServletResponse original              TeeServletOutputStream tee              ByteArrayOutputStream bos               public BufferedResponseWrapper HttpServletResponse response                    original   response                             public String getContent                     return bos toString                               public PrintWriter getWriter   throws IOException                   return original getWriter                               public ServletOutputStream getOutputStream   throws IOException                   if  tee    null                        bos   new ByteArrayOutputStream                        tee   new TeeServletOutputStream original getOutputStream                                bos                                     return tee                               Override             public String getCharacterEncoding                     return original getCharacterEncoding                                Override             public String getContentType                     return original getContentType                                Override             public void setCharacterEncoding String charset                    original setCharacterEncoding charset                               Override             public void setContentLength int len                    original setContentLength len                               Override             public void setContentLengthLong long l                    original setContentLengthLong l                               Override             public void setContentType String type                    original setContentType type                               Override             public void setBufferSize int size                    original setBufferSize size                               Override             public int getBufferSize                     return original getBufferSize                                Override             public void flushBuffer   throws IOException                   tee flush                                Override             public void resetBuffer                     original resetBuffer                                Override             public boolean isCommitted                     return original isCommitted                                Override             public void reset                     original reset                                Override             public void setLocale Locale loc                    original setLocale loc                               Override             public Locale getLocale                     return original getLocale                                Override             public void addCookie Cookie cookie                    original addCookie cookie                               Override             public boolean containsHeader String name                    return original containsHeader name                               Override             public String encodeURL String url                    return original encodeURL url                               Override             public String encodeRedirectURL String url                    return original encodeRedirectURL url                               SuppressWarnings  deprecation                Override             public String encodeUrl String url                    return original encodeUrl url                               SuppressWarnings  deprecation                Override             public String encodeRedirectUrl String url                    return original encodeRedirectUrl url                               Override             public void sendError int sc  String msg  throws IOException                   original sendError sc  msg                               Override             public void sendError int sc  throws IOException                   original sendError sc                               Override             public void sendRedirect String location  throws IOException                   original sendRedirect location                               Override             public void setDateHeader String name  long date                    original setDateHeader name  date                               Override             public void addDateHeader String name  long date                    original addDateHeader name  date                               Override             public void setHeader String name  String value                    original setHeader name  value                               Override             public void addHeader String name  String value                    original addHeader name  value                               Override             public void setIntHeader String name  int value                    original setIntHeader name  value                               Override             public void addIntHeader String name  int value                    original addIntHeader name  value                               Override             public void setStatus int sc                    original setStatus sc                               SuppressWarnings  deprecation                Override             public void setStatus int sc  String sm                    original setStatus sc  sm                               Override             public String getHeader String arg0                    return original getHeader arg0                               Override             public Collection lt String gt  getHeaderNames                     return original getHeaderNames                                Override             public Collection lt String gt  getHeaders String arg0                    return original getHeaders arg0                               Override             public int getStatus                     return original getStatus

User · Answer

You can also configure a custom Spring interceptor HandlerInterceptorAdapter for a simplified implementation of pre-only post-only interceptors    Component public class CustomHttpInterceptor extends HandlerInterceptorAdapter         Override     public boolean preHandle  final HttpServletRequest request  final HttpServletResponse response              final Object handler              throws Exception               Logs here          return super preHandle request  response  handler               Override     public void afterCompletion final HttpServletRequest request  final HttpServletResponse response              final Object handler  final Exception ex               Logs here           Then  you register as many interceptors as you want    Configuration public class WebMvcConfig implements WebMvcConfigurer         Autowired     CustomHttpInterceptor customHttpInterceptor        Override     public void addInterceptors InterceptorRegistry registry            registry addInterceptor customHttpInterceptor  addPathPatterns   endpoints                 Note  just like stated by  Robert  you need to pay attention to the specific implementations of HttpServletRequest and HttpServletResponse your application is using    For example  for apps using the ShallowEtagHeaderFilter  the response implementation would be a ContentCachingResponseWrapper  so you d have    Component public class CustomHttpInterceptor extends HandlerInterceptorAdapter        private static final Logger LOGGER   LoggerFactory getLogger CustomHttpInterceptor class        private static final int MAX PAYLOAD LENGTH   1000        Override     public void afterCompletion final HttpServletRequest request  final HttpServletResponse response              final Object handler  final Exception ex            final byte   contentAsByteArray     ContentCachingResponseWrapper  response  getContentAsByteArray             LOGGER info  Request body  n    getContentAsString contentAsByteArray  response getCharacterEncoding                 private String getContentAsString byte   buf  String charsetName            if  buf    null    buf length    0                return                        try               int length   Math min buf length  MAX PAYLOAD LENGTH                return new String buf  0  length  charsetName             catch  UnsupportedEncodingException ex                return  Unsupported Encoding

User · Answer

You could use javax servlet Filter if there wasn t a requirement to log java method that been executed   But with this requirement you have to access information stored in handlerMapping of DispatcherServlet  That said  you can override DispatcherServlet to accomplish logging of request response pair   Below is an example of idea that can be further enhanced and adopted to your needs   public class LoggableDispatcherServlet extends DispatcherServlet        private final Log logger   LogFactory getLog getClass           Override     protected void doDispatch HttpServletRequest request  HttpServletResponse response  throws Exception           if    request instanceof ContentCachingRequestWrapper                 request   new ContentCachingRequestWrapper request                     if    response instanceof ContentCachingResponseWrapper                 response   new ContentCachingResponseWrapper response                     HandlerExecutionChain handler   getHandler request            try               super doDispatch request  response             finally               log request  response  handler               updateResponse response                        private void log HttpServletRequest requestToCache  HttpServletResponse responseToCache  HandlerExecutionChain handler            LogMessage log   new LogMessage            log setHttpStatus responseToCache getStatus             log setHttpMethod requestToCache getMethod             log setPath requestToCache getRequestURI             log setClientIp requestToCache getRemoteAddr             log setJavaMethod handler toString             log setResponse getResponsePayload responseToCache            logger info log              private String getResponsePayload HttpServletResponse response            ContentCachingResponseWrapper wrapper   WebUtils getNativeResponse response  ContentCachingResponseWrapper class           if  wrapper    null                 byte   buf   wrapper getContentAsByteArray                if  buf length  gt  0                    int length   Math min buf length  5120                   try                       return new String buf  0  length  wrapper getCharacterEncoding                                       catch  UnsupportedEncodingException ex                           NOOP                                                   return   unknown               private void updateResponse HttpServletResponse response  throws IOException           ContentCachingResponseWrapper responseWrapper               WebUtils getNativeResponse response  ContentCachingResponseWrapper class           responseWrapper copyBodyToResponse               HandlerExecutionChain - contains the information about request handler   You then can register this dispatcher as following        Bean     public ServletRegistrationBean dispatcherRegistration             return new ServletRegistrationBean dispatcherServlet                 Bean name   DispatcherServletAutoConfiguration DEFAULT DISPATCHER SERVLET BEAN NAME      public DispatcherServlet dispatcherServlet             return new LoggableDispatcherServlet            And here s the sample of logs   http http   localhost 8090 settings test i g m s s LoggableDispatcherServlet        LogMessage httpStatus 500  path   error   httpMethod  GET   clientIp  127 0 0 1   javaMethod  HandlerExecutionChain with handler  public org springframework http ResponseEntity lt java util Map lt java lang String  java lang Object gt  gt  org springframework boot autoconfigure web BasicErrorController error javax servlet http HttpServletRequest   and 3 interceptors   arguments null  response    timestamp  1472475814077  status  500  error   Internal Server Error   exception   java lang RuntimeException   message   org springframework web util NestedServletException  Request processing failed  nested exception is java lang RuntimeException   path    settings test      http http   localhost 8090 settings params i g m s s LoggableDispatcherServlet        LogMessage httpStatus 200  path   settings httpParams   httpMethod  GET   clientIp  127 0 0 1   javaMethod  HandlerExecutionChain with handler  public x y z DTO x y z Controller params    and 3 interceptors   arguments null  response        http http   localhost 8090 123 i g m s s LoggableDispatcherServlet        LogMessage httpStatus 404  path   error   httpMethod  GET   clientIp  127 0 0 1   javaMethod  HandlerExecutionChain with handler  public org springframework http ResponseEntity lt java util Map lt java lang String  java lang Object gt  gt  org springframework boot autoconfigure web BasicErrorController error javax servlet http HttpServletRequest   and 3 interceptors   arguments null  response    timestamp  1472475840592  status  404  error   Not Found   message   Not Found   path    123       UPDATE  In case of errors Spring does automatic error handling  Therefore  BasicErrorController error is shown as request handler  If you want to preserve original request handler  then you can override this behavior at spring-webmvc-4 2 5 RELEASE-sources jar  org springframework web servlet DispatcherServlet java 971 before  processDispatchResult is called  to cache original handler

User · Answer

Please refer to below link for actual answer https   gist github com int128 e47217bebdb4c402b2ffa7cc199307ba  Made some changes from above referred solution   request and response will log in console and in file too if logger level is info  we can print either in console or file    Component public class LoggingFilter extends OncePerRequestFilter    private static final List lt MediaType gt  VISIBLE TYPES   Arrays asList          MediaType valueOf  text              MediaType APPLICATION FORM URLENCODED          MediaType APPLICATION JSON          MediaType APPLICATION XML          MediaType valueOf  application   json            MediaType valueOf  application   xml            MediaType MULTIPART FORM DATA            Logger log   LoggerFactory getLogger ReqAndResLoggingFilter class   private static final Path path   Paths get   home ramesh loggerReq txt    private static BufferedWriter writer   null   Override protected void doFilterInternal HttpServletRequest request  HttpServletResponse response  FilterChain filterChain  throws ServletException  IOException       try           writer   Files newBufferedWriter path  Charset forName  UTF-8         if  isAsyncDispatch request             filterChain doFilter request  response         else           doFilterWrapped wrapRequest request   wrapResponse response   filterChain              finally           writer close             protected void doFilterWrapped ContentCachingRequestWrapper request  ContentCachingResponseWrapper response  FilterChain filterChain  throws ServletException  IOException       try           beforeRequest request  response           filterChain doFilter request  response             finally           afterRequest request  response           response copyBodyToResponse             protected void beforeRequest ContentCachingRequestWrapper request  ContentCachingResponseWrapper response  throws IOException       if  log isInfoEnabled              logRequestHeader request  request getRemoteAddr        gt              protected void afterRequest ContentCachingRequestWrapper request  ContentCachingResponseWrapper response  throws IOException       if  log isInfoEnabled              logRequestBody request  request getRemoteAddr        gt             logResponse response  request getRemoteAddr        lt              private void logRequestHeader ContentCachingRequestWrapper request  String prefix  throws IOException       String queryString   request getQueryString        if  queryString    null            printLines prefix request getMethod   request getRequestURI             log info             prefix  request getMethod    request getRequestURI           else           printLines prefix request getMethod   request getRequestURI   queryString           log info                prefix  request getMethod    request getRequestURI    queryString             Collections list request getHeaderNames    forEach headerName - gt      Collections list request getHeaders headerName   forEach headerValue - gt      log info              prefix  headerName  headerValue         printLines prefix       printLines RequestContextHolder currentRequestAttributes   getSessionId         log info       prefix        log info   Session ID     RequestContextHolder currentRequestAttributes   getSessionId        private void printLines String    args  throws IOException        try       for String varArgs args                writer write varArgs               writer newLine                   catch IOException ex               ex printStackTrace              private void logRequestBody ContentCachingRequestWrapper request  String prefix        byte   content   request getContentAsByteArray        if  content length  gt  0            logContent content  request getContentType    request getCharacterEncoding    prefix            private void logResponse ContentCachingResponseWrapper response  String prefix  throws IOException       int status   response getStatus        printLines prefix  String valueOf status   HttpStatus valueOf status  getReasonPhrase         log info             prefix  status  HttpStatus valueOf status  getReasonPhrase         response getHeaderNames   forEach headerName - gt      response getHeaders headerName  forEach headerValue - gt      log info              prefix  headerName  headerValue         printLines prefix       log info       prefix       byte   content   response getContentAsByteArray        if  content length  gt  0            logContent content  response getContentType    response getCharacterEncoding    prefix            private void logContent byte   content  String contentType  String contentEncoding  String prefix        MediaType mediaType   MediaType valueOf contentType       boolean visible   VISIBLE TYPES stream   anyMatch visibleType - gt  visibleType includes mediaType        if  visible            try               String contentString   new String content  contentEncoding               Stream of contentString split   r n  r  n    forEach line - gt                    try                       printLines line                     catch  IOException e                           TODO Auto-generated catch block                     e printStackTrace                                                      log info          prefix  line              catch  UnsupportedEncodingException e                log info         bytes content    prefix  content length                   else            log info         bytes content    prefix  content length            private static ContentCachingRequestWrapper wrapRequest HttpServletRequest request        if  request instanceof ContentCachingRequestWrapper            return  ContentCachingRequestWrapper  request        else           return new ContentCachingRequestWrapper request            private static ContentCachingResponseWrapper wrapResponse HttpServletResponse response        if  response instanceof ContentCachingResponseWrapper            return  ContentCachingResponseWrapper  response        else           return new ContentCachingResponseWrapper response                Output in File   127 0 0 1  gt  POST  createUser 127 0 0 1  gt  session Id C0793464532E7F0C7154913CBA018B2B Request       name    asdasdas      birthDate    2018-06-21T17 11 15 679 0000    127 0 0 1  lt  200 OK 127 0 0 1  lt  Response    name   asdasdas   birthDate   2018-06-21T17 11 15 679 0000   id  4

User · Answer

After adding Actuators to the spring boot bassed application you have  trace endpoint available with latest requests informations  This endpoint is working based on TraceRepository and default implementation is InMemoryTraceRepository that saves last 100 calls  You can change this by implementing this interface by yourself and make it available as a Spring bean  For example to log all requests to log  and still use default implementation as a basic storage for serving info on  trace endpoint  I m using this kind of implementation    import org slf4j Logger  import org slf4j LoggerFactory  import org springframework boot actuate trace InMemoryTraceRepository  import org springframework boot actuate trace Trace  import org springframework boot actuate trace TraceRepository  import org springframework stereotype Component   import java util List  import java util Map     Component public class LoggingTraceRepository implements TraceRepository      private static final Logger LOG   LoggerFactory getLogger LoggingTraceRepository class     private final TraceRepository delegate   new InMemoryTraceRepository        Override   public List lt Trace gt  findAll         return delegate findAll            Override   public void add Map lt String  Object gt  traceInfo        LOG info traceInfo toString         this delegate add traceInfo           This traceInfo map contains basic informations about request and response in this kind of form   method GET  path  api hello John  headers  request  host localhost 8080  user-agent curl 7 51 0  accept       response  X-Application-Context application  Content-Type text plain charset UTF-8  Content-Length 10  Date Wed  29 Mar 2017 20 41 21 GMT  status 200     There is NO response content here   EDIT  Logging POST data  You can access POST data by overriding WebRequestTraceFilter  but don t think it is a good idea  e g  all uploaded file content will go to logs  Here is sample code  but don t use it   package info fingo nuntius acuate trace   import org apache commons io IOUtils  import org springframework boot actuate trace TraceProperties  import org springframework boot actuate trace TraceRepository  import org springframework boot actuate trace WebRequestTraceFilter  import org springframework stereotype Component   import javax servlet ServletException  import javax servlet http HttpServletRequest  import java io IOException  import java nio charset Charset  import java util LinkedHashMap  import java util Map    Component public class CustomWebTraceFilter extends WebRequestTraceFilter      public CustomWebTraceFilter TraceRepository repository  TraceProperties properties        super repository  properties         Override   protected Map lt String  Object gt  getTrace HttpServletRequest request        Map lt String  Object gt  trace   super getTrace request       String multipartHeader   request getHeader  content-type        if  multipartHeader    null  amp  amp  multipartHeader startsWith  multipart form-data              Map lt String  Object gt  parts   new LinkedHashMap lt  gt             try               request getParts   forEach                      part - gt                            try                               parts put part getName    IOUtils toString part getInputStream    Charset forName  UTF-8                                catch  IOException e                                e printStackTrace                                                                             catch  IOException   ServletException e                e printStackTrace                      if   parts isEmpty                  trace put  multipart-content-map   parts                       return trace

User · Answer

If you are seeing only part of your request payload  you need to call the setMaxPayloadLength function as it defaults to showing only 50 characters in your request body   Also  setting setIncludeHeaders to false is a good idea if you don t want to log your auth headers     Bean public CommonsRequestLoggingFilter requestLoggingFilter         CommonsRequestLoggingFilter loggingFilter   new CommonsRequestLoggingFilter        loggingFilter setIncludeClientInfo false       loggingFilter setIncludeQueryString false       loggingFilter setIncludePayload true       loggingFilter setIncludeHeaders false       loggingFilter setMaxPayloadLength 500       return loggingFilter

User · Answer

Don t write any Interceptors  Filters  Components  Aspects  etc   this is a very common problem and has been solved many times over    Spring Boot has a modules called Actuator  which provides HTTP request logging out of the box  There s an endpoint mapped to  trace  SB1 x  or  actuator httptrace  SB2 0   which will show you last 100 HTTP requests  You can customize it to log each request  or write to a DB     To get the endpoints you want  you ll need the spring-boot-starter-actuator dependency  and also to  whitelist  the endpoints you re looking for  and possibly setup or disable security for it   Also  where will this application run  Will you be using a PaaS  Hosting providers  Heroku for example  provide request logging as part of their service and you don t need to do any coding whatsoever then

User · Answer

Has there been any development with Actuator HTTP Trace since the initial question was posted i e  is there a way to enrich it with the response body  What about enriching it with custom metadata from MDC or from Spring-Sleuth or Zipkin  such as traceId and spanId  Also for me Actuator HTTP Trace didn t work Spring Boot 2 2 3  and I found the fix here  https   juplo de actuator-httptrace-does-not-work-with-spring-boot-2-2  pom xml  lt dependency gt     lt groupId gt org springframework boot    lt artifactId gt spring-boot-starter-actuator  lt  dependency gt   application properties management endpoints web exposure include httptrace  The fix   The simple fix for this problem is  to add a  Bean of type InMemoryHttpTraceRepository to your  Configuration-class    Bean public HttpTraceRepository htttpTraceRepository       return new InMemoryHttpTraceRepository       The Explanation   The cause of this problem is not a bug  but a legitimate change in the default configuration  Unfortunately  this change is not noted in the according section of the documentation  Instead it is burried in the Upgrade Notes for Spring Boot 2 2 The default-implementation stores the captured data in memory  Hence  it consumes much memory  without the user knowing  or even worse  needing it  This is especially undesirable in cluster environments  where memory is a precious good  And remember  Spring Boot was invented to simplify cluster deployments  That is  why this feature is now turned of by default and has to be turned on by the user explicitly  if needed

User · Answer

I had defined logging level in application properties to print requests responses  method url in the log file  logging level org springframework web DEBUG logging level org hibernate SQL INFO logging file D  log myapp log   I had used Spring Boot

User · Answer

hahn s answer required a bit of modification for it to work for me  but it is by far the most customizable thing I could get   It didn t work for me  probably because I also have a HandlerInterceptorAdapter     but I kept getting a bad response from the server in that version  Here s my modification of it   public class LoggableDispatcherServlet extends DispatcherServlet        private final Log logger   LogFactory getLog getClass           Override     protected void doDispatch HttpServletRequest request  HttpServletResponse response  throws Exception            long startTime   System currentTimeMillis            try               super doDispatch request  response             finally               log new ContentCachingRequestWrapper request   new ContentCachingResponseWrapper response                       System currentTimeMillis   - startTime                        private void log HttpServletRequest requestToCache  HttpServletResponse responseToCache  long timeTaken            int status   responseToCache getStatus            JsonObject jsonObject   new JsonObject            jsonObject addProperty  httpStatus   status           jsonObject addProperty  path   requestToCache getRequestURI             jsonObject addProperty  httpMethod   requestToCache getMethod             jsonObject addProperty  timeTakenMs   timeTaken           jsonObject addProperty  clientIP   requestToCache getRemoteAddr             if  status  gt  299                String requestBody   null              try                   requestBody   requestToCache getReader   lines   collect Collectors joining System lineSeparator                    catch  IOException e                    e printStackTrace                              jsonObject addProperty  requestBody   requestBody               jsonObject addProperty  requestParams   requestToCache getQueryString                 jsonObject addProperty  tokenExpiringHeader                       responseToCache getHeader ResponseHeaderModifierInterceptor HEADER TOKEN EXPIRING                      logger info jsonObject

User · Answer

Currently Spring Boot has the Actuator feature to get the logs of requests and responses  But you can also get the logs using Aspect AOP   Aspect provides you with annotations like   Before    AfterReturning   AfterThrowing etc   Before logs the request   AfterReturning logs the response and  AfterThrowing logs the error message  You may not need all endpoints  log  so you can apply some filters on the packages  Here are some examples  For Request   Before  quot within your package where endpoints are     quot       public void endpointBefore JoinPoint p            if  log isTraceEnabled                  log trace p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot  START quot                Object   signatureArgs   p getArgs                  ObjectMapper mapper   new ObjectMapper                mapper enable SerializationFeature INDENT OUTPUT               try                    if  signatureArgs 0     null                        log trace  quot  nRequest object   n quot    mapper writeValueAsString signatureArgs 0                                     catch  JsonProcessingException e                                   Here  Before  quot within your package where endpoints are     quot   has the package path  All endpoints within this package will generate the log  For Response   AfterReturning value     quot within your package where endpoints are     quot                returning    quot returnValue quot       public void endpointAfterReturning JoinPoint p  Object returnValue            if  log isTraceEnabled                  ObjectMapper mapper   new ObjectMapper                mapper enable SerializationFeature INDENT OUTPUT               try                   log trace  quot  nResponse object   n quot    mapper writeValueAsString returnValue                  catch  JsonProcessingException e                    System out println e getMessage                               log trace p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot  END quot                     Here  AfterReturning  quot within your package where endpoints are     quot   has the package path  All endpoints within this package will generate the log  Also Object returnValue contains the response  For Exception   AfterThrowing pointcut     quot within your package where endpoints are     quot    throwing    quot e quot   public void endpointAfterThrowing JoinPoint p  Exception e  throws DmoneyException       if  log isTraceEnabled              System out println e getMessage              e printStackTrace              log error p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot   quot    e getMessage              Here  AfterThrowing pointcut     quot within your package where endpoints are     quot    throwing    quot e quot    has the package path  All endpoints within this package will generate the log  Also Exception e contains the error response  Here is the full code  import com fasterxml jackson core JsonProcessingException  import com fasterxml jackson databind ObjectMapper  import com fasterxml jackson databind SerializationFeature  import org apache log4j Logger  import org aspectj lang JoinPoint  import org aspectj lang annotation AfterReturning  import org aspectj lang annotation AfterThrowing  import org aspectj lang annotation Aspect  import org aspectj lang annotation Before  import org springframework core annotation Order  import org springframework stereotype Component    Aspect  Order 1   Component  ConditionalOnExpression  quot   endpoint aspect enabled true  quot   public class EndpointAspect       static Logger log   Logger getLogger EndpointAspect class         Before  quot within your package where is endpoint     quot       public void endpointBefore JoinPoint p            if  log isTraceEnabled                  log trace p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot  START quot                Object   signatureArgs   p getArgs                  ObjectMapper mapper   new ObjectMapper                mapper enable SerializationFeature INDENT OUTPUT               try                    if  signatureArgs 0     null                        log trace  quot  nRequest object   n quot    mapper writeValueAsString signatureArgs 0                                     catch  JsonProcessingException e                                        AfterReturning value     quot within your package where is endpoint     quot                returning    quot returnValue quot       public void endpointAfterReturning JoinPoint p  Object returnValue            if  log isTraceEnabled                  ObjectMapper mapper   new ObjectMapper                mapper enable SerializationFeature INDENT OUTPUT               try                   log trace  quot  nResponse object   n quot    mapper writeValueAsString returnValue                  catch  JsonProcessingException e                    System out println e getMessage                               log trace p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot  END quot                           AfterThrowing pointcut     quot within your package where is endpoint     quot    throwing    quot e quot       public void endpointAfterThrowing JoinPoint p  Exception e  throws Exception           if  log isTraceEnabled                  System out println e getMessage                  e printStackTrace                  log error p getTarget   getClass   getSimpleName      quot   quot    p getSignature   getName      quot   quot    e getMessage                        Here  using  ConditionalOnExpression  quot   endpoint aspect enabled true  quot   you can enable disable the log  just add endpoint aspect enabled true into the application property and control the log More info about AOP visit here  Spring docs about AOP Sample article about AOP

User · Answer

In order to log requests that result in 400 only   import javax servlet FilterChain  import javax servlet ServletException  import javax servlet http HttpServletRequest  import javax servlet http HttpServletResponse  import javax servlet http HttpSession   import org apache commons io FileUtils  import org springframework http HttpStatus  import org springframework http server ServletServerHttpRequest  import org springframework stereotype Component  import org springframework util StringUtils  import org springframework web filter AbstractRequestLoggingFilter  import org springframework web filter OncePerRequestFilter  import org springframework web util ContentCachingRequestWrapper  import org springframework web util WebUtils          Implementation is partially copied from   link AbstractRequestLoggingFilter  and modified to output request information only if request resulted in 400     Unfortunately   link AbstractRequestLoggingFilter  is not smart enough to expose   link HttpServletResponse  value in afterRequest   method       Component public class RequestLoggingFilter extends OncePerRequestFilter        public static final String DEFAULT AFTER MESSAGE PREFIX    After request          public static final String DEFAULT AFTER MESSAGE SUFFIX             private final boolean includeQueryString   true      private final boolean includeClientInfo   true      private final boolean includeHeaders   true      private final boolean includePayload   true       private final int maxPayloadLength    int   2   FileUtils ONE MB        private final String afterMessagePrefix   DEFAULT AFTER MESSAGE PREFIX       private final String afterMessageSuffix   DEFAULT AFTER MESSAGE SUFFIX                  The default value is  false  so that the filter may log a  before  message        at the start of request processing and an  after  message at the end from        when the last asynchronously dispatched thread is exiting               Override     protected boolean shouldNotFilterAsyncDispatch             return false              Override     protected void doFilterInternal final HttpServletRequest request  final HttpServletResponse response  final FilterChain filterChain              throws ServletException  IOException            final boolean isFirstRequest    isAsyncDispatch request           HttpServletRequest requestToUse   request           if  includePayload  amp  amp  isFirstRequest  amp  amp    request instanceof ContentCachingRequestWrapper                 requestToUse   new ContentCachingRequestWrapper request  maxPayloadLength                      final boolean shouldLog   shouldLog requestToUse            try               filterChain doFilter requestToUse  response             finally               if  shouldLog  amp  amp   isAsyncStarted requestToUse                     afterRequest requestToUse  response  getAfterMessage requestToUse                                       private String getAfterMessage final HttpServletRequest request            return createMessage request  this afterMessagePrefix  this afterMessageSuffix              private String createMessage final HttpServletRequest request  final String prefix  final String suffix            final StringBuilder msg   new StringBuilder            msg append prefix           msg append  uri    append request getRequestURI              if  includeQueryString                final String queryString   request getQueryString                if  queryString    null                    msg append      append queryString                                    if  includeClientInfo                final String client   request getRemoteAddr                if  StringUtils hasLength client                     msg append   client    append client                             final HttpSession session   request getSession false               if  session    null                    msg append   session    append session getId                               final String user   request getRemoteUser                if  user    null                    msg append   user    append user                                    if  includeHeaders                msg append   headers    append new ServletServerHttpRequest request  getHeaders                        if  includeHeaders                final ContentCachingRequestWrapper wrapper   WebUtils getNativeRequest request  ContentCachingRequestWrapper class               if  wrapper    null                    final byte   buf   wrapper getContentAsByteArray                    if  buf length  gt  0                        final int length   Math min buf length  maxPayloadLength                       String payload                      try                           payload   new String buf  0  length  wrapper getCharacterEncoding                           catch  final UnsupportedEncodingException ex                            payload     unknown                                              msg append   payload    append payload                                                     msg append suffix           return msg toString               private boolean shouldLog final HttpServletRequest request            return true             private void afterRequest final HttpServletRequest request  final HttpServletResponse response  final String message            if  response getStatus      HttpStatus BAD REQUEST value                  logger warn message

User · Answer

If you dont mind trying Spring AOP  this is something I have been exploring for logging purposes and it works pretty well for me  It wont log requests that have not been defined and failed request attempts though   Add these three dependencies  spring-aop  aspectjrt  aspectjweaver   Add this to your xml config file  lt aop aspectj-autoproxy  gt   Create an annotation which can be used as a pointcut    Retention RetentionPolicy RUNTIME   Target  ElementType METHOD ElementType TYPE   public  interface EnableLogging   ActionType actionType        Now annotate all your rest API methods which you want to log   EnableLogging actionType   ActionType SOME EMPLOYEE ACTION   Override public Response getEmployees RequestDto req  final String param            Now on to the Aspect  component-scan the package which this class is in    Aspect  Component public class Aspects     AfterReturning pointcut    execution  co xyz aspect EnableLogging           amp  amp   annotation enableLogging   amp  amp  args reqArg  reqArg1       returning    result   public void auditInfo JoinPoint joinPoint  Object result  EnableLogging enableLogging  Object reqArg  String reqArg1         HttpServletRequest request     ServletRequestAttributes  RequestContextHolder currentRequestAttributes                 getRequest         if  result instanceof Response            Response responseObj    Response  result       String requestUrl   request getScheme             request getServerName                           request getServerPort     request getContextPath     request getRequestURI                           request getQueryString     String clientIp   request getRemoteAddr    String clientRequest   reqArg toString    int httpResponseStatus   responseObj getStatus    responseObj getEntity       Can log whatever stuff from here in a single spot       AfterThrowing pointcut    execution  co xyz aspect EnableLogging           amp  amp   annotation enableLogging   amp  amp  args reqArg  reqArg1       throwing  exception   public void auditExceptionInfo JoinPoint joinPoint  Throwable exception  EnableLogging enableLogging  Object reqArg  String reqArg1         HttpServletRequest request     ServletRequestAttributes  RequestContextHolder currentRequestAttributes                 getRequest         String requestUrl   request getScheme             request getServerName               request getServerPort     request getContextPath     request getRequestURI               request getQueryString         exception getMessage        exception getCause        exception printStackTrace        exception getLocalizedMessage           Can log whatever exceptions  requests  etc from here in a single spot                AfterReturning advice runs when a matched method execution returns   normally        AfterThrowing advice runs when a matched method execution exits by   throwing an exception    If you want to read in detail read through this  http   docs spring io spring docs current spring-framework-reference html aop html

User · Answer

if you use Tomcat in your boot app here is org apache catalina filters RequestDumperFilter in a class path for you   but it will not provide you  with exceptions in single place

User · Answer

the code pasted below works with my tests and can be downloaded from my  github project  1   sharing after applying a solution based on that on a production project    Configuration public class LoggingFilter extends GenericFilterBean                   It s important that you actually register your filter this way rather then just annotating it        as  Component as you need to be able to set for which  DispatcherType s to enable the filter         see point  1                   return              Bean     public FilterRegistrationBean lt LoggingFilter gt  initFilter             FilterRegistrationBean lt LoggingFilter gt  registrationBean   new FilterRegistrationBean lt  gt             registrationBean setFilter new LoggingFilter                  1  make sure you sett all dispatcher types if you want the filter to log upon         registrationBean setDispatcherTypes EnumSet allOf DispatcherType class                 2  this should put your filter above any other filter         registrationBean setOrder Ordered HIGHEST PRECEDENCE            return registrationBean              Override     public void doFilter ServletRequest request  ServletResponse response  FilterChain chain              throws IOException  ServletException            ContentCachingRequestWrapper wreq                new ContentCachingRequestWrapper                   HttpServletRequest  request            ContentCachingResponseWrapper wres                new ContentCachingResponseWrapper                   HttpServletResponse  response            try                   let it be                 chain doFilter wreq  wres                   makes sure that the input is read  e g  in 404 it may not be              while  wreq getInputStream   read    gt   0                System out printf      REQUEST n s n    end request n                       new String wreq getContentAsByteArray                      Do whatever logging you wish here  in this case I m writing request                 and response to system out which is probably not what you wish to do             System out printf      RESPONSE n s n    end response n                       new String wres getContentAsByteArray                      this is specific of the  ContentCachingResponseWrapper  we are relying on                  make sure you call it after you read the content from the response             wres copyBodyToResponse                    One more point  in case of redirect this will be called twice  beware to handle that                somewhat            catch  Throwable t                   Do whatever logging you whish here  too                here you should also be logging the error                throw t

User · Answer

If somebody still need it here is simple implementation with Spring HttpTrace Actuator  But as they have told upper it doesn t log bodies   import lombok extern slf4j Slf4j  import org apache commons lang3 builder ToStringBuilder  import org springframework boot actuate trace http HttpTrace  import org springframework boot actuate trace http InMemoryHttpTraceRepository  import org springframework stereotype Repository    Slf4j  Repository public class LoggingInMemoryHttpTraceRepository extends InMemoryHttpTraceRepository       public void add HttpTrace trace            super add trace           log info  Trace     ToStringBuilder reflectionToString trace            log info  Request     ToStringBuilder reflectionToString trace getRequest              log info  Response     ToStringBuilder reflectionToString trace getResponse

User · Answer

Spring already provides a filter that does this job  Add following bean to your config   Bean public CommonsRequestLoggingFilter requestLoggingFilter         CommonsRequestLoggingFilter loggingFilter   new CommonsRequestLoggingFilter        loggingFilter setIncludeClientInfo true       loggingFilter setIncludeQueryString true       loggingFilter setIncludePayload true       loggingFilter setMaxPayloadLength 64000       return loggingFilter      Don t forget to change log level of org springframework web filter CommonsRequestLoggingFilter to DEBUG

User · Answer

In order to log all the requests with input parameters and body  we can use filters and interceptors  But while using a filter or interceptor  we cannot print the request body multiple times  The better way is we can use spring-AOP  By using this we can decouple the logging mechanism from the application  AOP can be used for logging Input and output of each method in the application   My solution is    import org aspectj lang ProceedingJoinPoint   import org aspectj lang annotation Around   import org aspectj lang annotation Aspect   import org aspectj lang annotation Pointcut   import org aspectj lang reflect CodeSignature   import org slf4j Logger   import org slf4j LoggerFactory   import org springframework stereotype Component   import com fasterxml jackson databind ObjectMapper    Aspect   Component public class LoggingAdvice   private static final Logger logger    LoggerFactory getLogger LoggingAdvice class      here we can provide any methodName  packageName  className   Pointcut value    execution   com package name               public void myPointcut          Around  myPointcut     public Object applicationLogger ProceedingJoinPoint pjt  throws Throwable       ObjectMapper mapper   new ObjectMapper        String methodName   pjt getSignature   getName        String className   pjt getTarget   getClass   toString        String inputParams   this getInputArgs pjt  mapper       logger info  method invoked from     className           methodName    --Request Payload      inputParams       Object object   pjt proceed        try           logger info  Response Object---    mapper writeValueAsString object          catch  Exception e              return object     private String getInputArgs ProceedingJoinPoint pjt ObjectMapper mapper        Object   array   pjt getArgs        CodeSignature signature    CodeSignature  pjt getSignature         StringBuilder sb   new StringBuilder        sb append           int i   0      String   parameterNames   signature getParameterNames        int maxArgs   parameterNames length      for  String name   signature getParameterNames              sb append      append name  append               try               sb append mapper writeValueAsString array i    append                   if i    maxArgs -1                     sb append                               catch  Exception e                sb append                          i              return sb append      toString

User · Answer

As suggested previously  Logbook is just about perfect for this  but I did have a little trouble setting it up when using Java modules  due to a split package between logbook-api and logbook-core  For my Gradle   Spring Boot project  I needed build gradle dependencies       compileOnly group   org zalando   name   logbook-api   version   2 4 1      runtimeOnly group   org zalando   name   logbook-spring-boot-starter   version   2 4 1                logback-spring xml  lt configuration gt       lt  -- HTTP Requests and Responses -- gt       lt logger name  quot org zalando logbook quot  level  quot trace quot    gt   lt  configuration gt

User · Answer

If you have Spring boot Config server configured then just enable Debug logger for class     Http11InputBuffer Http11InputBuffer java   Debugs will log all the requests and  responses for every request

User · Answer

Here my solution  Spring 2 0 x   Add the maven dependency    lt dependency gt       lt groupId gt org springframework boot lt  groupId gt       lt artifactId gt spring-boot-starter-actuator lt  artifactId gt   lt  dependency gt    Edit the application properties and add the following line   management endpoints web exposure include      Once your spring boot application is started you can track the latest 100 http requests by calling this url   http   localhost 8070 actuator httptrace

[java] Spring Boot - How to log all requests and responses with exceptions in single place?

Notes

Examples related to java

Examples related to spring

Examples related to logging

Examples related to spring-rest