[c] Error handling in C code

What do you consider "best practice" when it comes to error handling errors in a consistent way in a C library.

There are two ways I've been thinking of:

Always return error code. A typical function would look like this:

MYAPI_ERROR getObjectSize(MYAPIHandle h, int* returnedSize);

The always provide an error pointer approach:

int getObjectSize(MYAPIHandle h, MYAPI_ERROR* returnedError);

When using the first approach it's possible to write code like this where the error handling check is directly placed on the function call:

int size;
if(getObjectSize(h, &size) != MYAPI_SUCCESS) {
  // Error handling
}

Which looks better than the error handling code here.

MYAPIError error;
int size;
size = getObjectSize(h, &error);
if(error != MYAPI_SUCCESS) {
    // Error handling
}

However, I think using the return value for returning data makes the code more readable, It's obvious that something was written to the size variable in the second example.

Do you have any ideas on why I should prefer any of those approaches or perhaps mix them or use something else? I'm not a fan of global error states since it tends to make multi threaded use of the library way more painful.

EDIT: C++ specific ideas on this would also be interesting to hear about as long as they are not involving exceptions since it's not an option for me at the moment...

I use the first approach whenever I create a library. There are several advantages of using a typedef'ed enum as a return code.

• If the function returns a more complicated output such as an array and it's length you do not need to create arbitrary structures to return.

  rc = func(..., int **return_array, size_t *array_length);
  

• It allows for simple, standardized error handling.

  if ((rc = func(...)) != API_SUCCESS) {
     /* Error Handling */
  }
  

• It allows for simple error handling in the library function.

  /* Check for valid arguments */
  if (NULL == return_array || NULL == array_length)
      return API_INVALID_ARGS;
  

• Using a typedef'ed enum also allows for the enum name to be visible in the debugger. This allows for easier debugging without the need to constantly consult a header file. Having a function to translate this enum into a string is helpful as well.

The most important issue regardless of approach used is to be consistent. This applies to function and argument naming, argument ordering and error handling.

I definitely prefer the first solution :

int size;
if(getObjectSize(h, &size) != MYAPI_SUCCESS) {
  // Error handling
}

i would slightly modify it, to:

int size;
MYAPIError rc;

rc = getObjectSize(h, &size)
if ( rc != MYAPI_SUCCESS) {
  // Error handling
}

In additional i will never mix legitimate return value with error even if currently the scope of function allowing you to do so, you never know which way function implementation will go in the future.

And if we already talking about error handling i would suggest goto Error; as error handling code, unless some undo function can be called to handle error handling correctly.

First approach is better IMHO:

• It's easier to write function that way. When you notice an error in the middle of the function you just return an error value. In second approach you need to assign error value to one of the parameters and then return something.... but what would you return - you don't have correct value and you don't return error value.
• it's more popular so it will be easier to understand, maintain

When I write programs, during initialization, I usually spin off a thread for error handling, and initialize a special structure for errors, including a lock. Then, when I detect an error, through return values, I enter in the info from the exception into the structure and send a SIGIO to the exception handling thread, then see if I can't continue execution. If I can't, I send a SIGURG to the exception thread, which stops the program gracefully.

I personally prefer the former approach (returning an error indicator).

Where necessary the return result should just indicate that an error occurred, with another function being used to find out the exact error.

In your getSize() example I'd consider that sizes must always be zero or positive, so returning a negative result can indicate an error, much like UNIX system calls do.

I can't think of any library that I've used that goes for the latter approach with an error object passed in as a pointer. stdio, etc all go with a return value.

I use the first approach whenever I create a library. There are several advantages of using a typedef'ed enum as a return code.

• If the function returns a more complicated output such as an array and it's length you do not need to create arbitrary structures to return.

  rc = func(..., int **return_array, size_t *array_length);
  

• It allows for simple, standardized error handling.

  if ((rc = func(...)) != API_SUCCESS) {
     /* Error Handling */
  }
  

• It allows for simple error handling in the library function.

  /* Check for valid arguments */
  if (NULL == return_array || NULL == array_length)
      return API_INVALID_ARGS;
  

• Using a typedef'ed enum also allows for the enum name to be visible in the debugger. This allows for easier debugging without the need to constantly consult a header file. Having a function to translate this enum into a string is helpful as well.

The most important issue regardless of approach used is to be consistent. This applies to function and argument naming, argument ordering and error handling.

Second approach lets the compiler produce more optimized code, because when address of a variable is passed to a function, the compiler cannot keep its value in register(s) during subsequent calls to other functions. The completion code usually is used only once, just after the call, whereas "real" data returned from the call may be used more often

In addition to what has been said, prior to returning your error code, fire off an assert or similar diagnostic when an error is returned, as it will make tracing a lot easier. The way I do this is to have a customised assert that still gets compiled in at release but only gets fired when the software is in diagnostics mode, with an option to silently report to a log file or pause on screen.

I personally return error codes as negative integers with no_error as zero , but it does leave you with the possible following bug

if (MyFunc())
 DoSomething();

An alternative is have a failure always returned as zero, and use a LastError() function to provide details of the actual error.

EDIT:If you need access only to the last error, and you don't work in multithreaded environment.

You can return only true/false (or some kind of #define if you work in C and don't support bool variables), and have a global Error buffer that will hold the last error:

int getObjectSize(MYAPIHandle h, int* returnedSize);
MYAPI_ERROR LastError;
MYAPI_ERROR* getLastError() {return LastError;};
#define FUNC_SUCCESS 1
#define FUNC_FAIL 0

if(getObjectSize(h, &size) != FUNC_SUCCESS ) {
    MYAPI_ERROR* error = getLastError();
    // error handling
}

I prefer error handling in C using the following technique:

struct lnode *insert(char *data, int len, struct lnode *list) {
    struct lnode *p, *q;
    uint8_t good;
    struct {
            uint8_t alloc_node : 1;
            uint8_t alloc_str : 1;
    } cleanup = { 0, 0 };

   // allocate node.
    p = (struct lnode *)malloc(sizeof(struct lnode));
    good = cleanup.alloc_node = (p != NULL);

   // good? then allocate str
    if (good) {
            p->str = (char *)malloc(sizeof(char)*len);
            good = cleanup.alloc_str = (p->str != NULL);
    }

   // good? copy data
    if(good) {
            memcpy ( p->str, data, len );
    }

   // still good? insert in list
    if(good) {
            if(NULL == list) {
                    p->next = NULL;
                    list = p;
            } else {
                    q = list;
                    while(q->next != NULL && good) {
                            // duplicate found--not good
                            good = (strcmp(q->str,p->str) != 0);
                            q = q->next;
                    }
                    if (good) {
                            p->next = q->next;
                            q->next = p;
                    }
            }
    }

   // not-good? cleanup.
    if(!good) {
            if(cleanup.alloc_str)   free(p->str);
            if(cleanup.alloc_node)  free(p);
    }

   // good? return list or else return NULL
    return (good ? list : NULL);
}

Source: http://blog.staila.com/?p=114

In addition to what has been said, prior to returning your error code, fire off an assert or similar diagnostic when an error is returned, as it will make tracing a lot easier. The way I do this is to have a customised assert that still gets compiled in at release but only gets fired when the software is in diagnostics mode, with an option to silently report to a log file or pause on screen.

I personally return error codes as negative integers with no_error as zero , but it does leave you with the possible following bug

if (MyFunc())
 DoSomething();

An alternative is have a failure always returned as zero, and use a LastError() function to provide details of the actual error.

I personally prefer the former approach (returning an error indicator).

Where necessary the return result should just indicate that an error occurred, with another function being used to find out the exact error.

In your getSize() example I'd consider that sizes must always be zero or positive, so returning a negative result can indicate an error, much like UNIX system calls do.

I can't think of any library that I've used that goes for the latter approach with an error object passed in as a pointer. stdio, etc all go with a return value.

In addition to what has been said, prior to returning your error code, fire off an assert or similar diagnostic when an error is returned, as it will make tracing a lot easier. The way I do this is to have a customised assert that still gets compiled in at release but only gets fired when the software is in diagnostics mode, with an option to silently report to a log file or pause on screen.

I personally return error codes as negative integers with no_error as zero , but it does leave you with the possible following bug

if (MyFunc())
 DoSomething();

An alternative is have a failure always returned as zero, and use a LastError() function to provide details of the actual error.

I prefer error handling in C using the following technique:

struct lnode *insert(char *data, int len, struct lnode *list) {
    struct lnode *p, *q;
    uint8_t good;
    struct {
            uint8_t alloc_node : 1;
            uint8_t alloc_str : 1;
    } cleanup = { 0, 0 };

   // allocate node.
    p = (struct lnode *)malloc(sizeof(struct lnode));
    good = cleanup.alloc_node = (p != NULL);

   // good? then allocate str
    if (good) {
            p->str = (char *)malloc(sizeof(char)*len);
            good = cleanup.alloc_str = (p->str != NULL);
    }

   // good? copy data
    if(good) {
            memcpy ( p->str, data, len );
    }

   // still good? insert in list
    if(good) {
            if(NULL == list) {
                    p->next = NULL;
                    list = p;
            } else {
                    q = list;
                    while(q->next != NULL && good) {
                            // duplicate found--not good
                            good = (strcmp(q->str,p->str) != 0);
                            q = q->next;
                    }
                    if (good) {
                            p->next = q->next;
                            q->next = p;
                    }
            }
    }

   // not-good? cleanup.
    if(!good) {
            if(cleanup.alloc_str)   free(p->str);
            if(cleanup.alloc_node)  free(p);
    }

   // good? return list or else return NULL
    return (good ? list : NULL);
}

Source: http://blog.staila.com/?p=114

Use setjmp.

http://en.wikipedia.org/wiki/Setjmp.h

http://aszt.inf.elte.hu/~gsd/halado_cpp/ch02s03.html

http://www.di.unipi.it/~nids/docs/longjump_try_trow_catch.html

#include <setjmp.h>
#include <stdio.h>

jmp_buf x;

void f()
{
    longjmp(x,5); // throw 5;
}

int main()
{
    // output of this program is 5.

    int i = 0;

    if ( (i = setjmp(x)) == 0 )// try{
    {
        f();
    } // } --> end of try{
    else // catch(i){
    {
        switch( i )
        {
        case  1:
        case  2:
        default: fprintf( stdout, "error code = %d\n", i); break;
        }
    } // } --> end of catch(i){
    return 0;
}

#include <stdio.h>
#include <setjmp.h>

#define TRY do{ jmp_buf ex_buf__; if( !setjmp(ex_buf__) ){
#define CATCH } else {
#define ETRY } }while(0)
#define THROW longjmp(ex_buf__, 1)

int
main(int argc, char** argv)
{
   TRY
   {
      printf("In Try Statement\n");
      THROW;
      printf("I do not appear\n");
   }
   CATCH
   {
      printf("Got Exception!\n");
   }
   ETRY;

   return 0;
}

Second approach lets the compiler produce more optimized code, because when address of a variable is passed to a function, the compiler cannot keep its value in register(s) during subsequent calls to other functions. The completion code usually is used only once, just after the call, whereas "real" data returned from the call may be used more often

Here's a simple program to demonstrate the first 2 bullets of Nils Pipenbrinck's answer here.

His first 2 bullets are:

• store all possible error-states in one typedef'ed enum and use it in your lib. Don't just return ints or even worse, mix ints or different enumerations with return-codes.

• provide a function that converts errors into something human readable. Can be simple. Just error-enum in, const char* out.

Assume you have written a module named mymodule. First, in mymodule.h, you define your enum-based error codes, and you write some error strings which correspond to these codes. Here I am using an array of C strings (char *), which only works well if your first enum-based error code has value 0, and you don't manipulate the numbers thereafter. If you do use error code numbers with gaps or other starting values, you'll simply have to change from using a mapped C-string array (as I do below) to using a function which uses a switch statement or if / else if statements to map from enum error codes to printable C strings (which I don't demonstrate). The choice is yours.

mymodule.h

/// @brief Error codes for library "mymodule"
typedef enum mymodule_error_e
{
    /// No error
    MYMODULE_ERROR_OK = 0,
    
    /// Invalid arguments (ex: NULL pointer where a valid pointer is required)
    MYMODULE_ERROR_INVARG,

    /// Out of memory (RAM)
    MYMODULE_ERROR_NOMEM,

    /// Make up your error codes as you see fit
    MYMODULE_ERROR_MYERROR, 

    // etc etc
    
    /// Total # of errors in this list (NOT AN ACTUAL ERROR CODE);
    /// NOTE: that for this to work, it assumes your first error code is value 0 and you let it naturally 
    /// increment from there, as is done above, without explicitly altering any error values above
    MYMODULE_ERROR_COUNT,
} mymodule_error_t;

// Array of strings to map enum error types to printable strings
// - see important NOTE above!
const char* const MYMODULE_ERROR_STRS[] = 
{
    "MYMODULE_ERROR_OK",
    "MYMODULE_ERROR_INVARG",
    "MYMODULE_ERROR_NOMEM",
    "MYMODULE_ERROR_MYERROR",
};

// To get a printable error string
const char* mymodule_error_str(mymodule_error_t err);

// Other functions in mymodule
mymodule_error_t mymodule_func1(void);
mymodule_error_t mymodule_func2(void);
mymodule_error_t mymodule_func3(void);

mymodule.c contains my mapping function to map from enum error codes to printable C strings:

mymodule.c

#include <stdio.h>

/// @brief      Function to get a printable string from an enum error type
/// @param[in]  err     a valid error code for this module
/// @return     A printable C string corresponding to the error code input above, or NULL if an invalid error code
///             was passed in
const char* mymodule_error_str(mymodule_error_t err)
{
    const char* err_str = NULL;

    // Ensure error codes are within the valid array index range
    if (err >= MYMODULE_ERROR_COUNT)
    {
        goto done;
    }

    err_str = MYMODULE_ERROR_STRS[err];

done:
    return err_str;
}

// Let's just make some empty dummy functions to return some errors; fill these in as appropriate for your 
// library module

mymodule_error_t mymodule_func1(void)
{
    return MYMODULE_ERROR_OK;
}

mymodule_error_t mymodule_func2(void)
{
    return MYMODULE_ERROR_INVARG;
}

mymodule_error_t mymodule_func3(void)
{
    return MYMODULE_ERROR_MYERROR;
}

main.c contains a test program to demonstrate calling some functions and printing some error codes from them:

main.c

#include <stdio.h>

int main()
{
    printf("Demonstration of enum-based error codes in C (or C++)\n");

    printf("err code from mymodule_func1() = %s\n", mymodule_error_str(mymodule_func1()));
    printf("err code from mymodule_func2() = %s\n", mymodule_error_str(mymodule_func2()));
    printf("err code from mymodule_func3() = %s\n", mymodule_error_str(mymodule_func3()));

    return 0;
}

Output:

Demonstration of enum-based error codes in C (or C++)
err code from mymodule_func1() = MYMODULE_ERROR_OK
err code from mymodule_func2() = MYMODULE_ERROR_INVARG
err code from mymodule_func3() = MYMODULE_ERROR_MYERROR

References:

You can run this code yourself here: https://onlinegdb.com/ByEbKLupS.

I ran into this Q&A a number of times, and wanted to contribute a more comprehensive answer. I think the best way to think about this is how to return errors to caller, and what you return.

How

There are 3 ways to return information from a function:

1. Return Value
2. Out Argument(s)
3. Out of Band, that includes non-local goto (setjmp/longjmp), file or global scoped variables, file system etc.

Return Value

You can only return value is a single object, however, it can be an arbitrary complex. Here is an example of an error returning function:

  enum error hold_my_beer();

One benefit of return values is that it allows chaining of calls for less intrusive error handling:

  !hold_my_beer() &&
  !hold_my_cigarette() &&
  !hold_my_pants() ||
  abort();

This not just about readability, but may also allow processing an array of such function pointers in a uniform way.

Out Argument(s)

You can return more via more than one object via arguments, but best practice does suggest to keep the total number of arguments low (say, <=4):

void look_ma(enum error *e, char *what_broke);

enum error e;
look_ma(e);
if(e == FURNITURE) {
  reorder(what_broke);
} else if(e == SELF) {
  tell_doctor(what_broke);
}

Out of Band

With setjmp() you define a place and how you want to handle an int value, and you transfer control to that location via a longjmp(). See Practical usage of setjmp and longjmp in C.

What

1. Indicator
2. Code
3. Object
4. Callback

Indicator

An error indicator only tells you that there is a problem but nothing about the nature of said problem:

struct foo *f = foo_init();
if(!f) {
  /// handle the absence of foo
}

This is the least powerful way for a function to communicate error state, however, perfect if caller cannot respond to the error in a graduated manner anyways.

Code

An error code tells the caller about the nature of the problem, and may allow for a suitable response (from the above). It can be return value, or like the look_ma() example above an error argument.

Object

With an error object, the caller can be informed about arbitrary complicated issues. For example, an error code and a suitable human readable message. It can also inform the caller that multiple things went wrong, or an error per item when processing a collection:

struct collection friends;
enum error *e = malloc(c.size * sizeof(enum error));
...
ask_for_favor(friends, reason);
for(int i = 0; i < c.size; i++) {
   if(reason[i] == NOT_FOUND) find(friends[i]);
}

Instead of pre-allocating the error array, you can also (re)allocate it dynamically as needed of course.

Callback

Callback is the most powerful way to handle errors, as you can tell the function what behavior you would like to see happen when something goes wrong. A callback argument can be added to each function, or if customization uis only required per instance of a struct like this:

 struct foo {
    ...
    void (error_handler)(char *);
 };

 void default_error_handler(char *message) { 
    assert(f);
    printf("%s", message);
 }

 void foo_set_error_handler(struct foo *f, void (*eh)(char *)) {
    assert(f);
    f->error_handler = eh;
 }

 struct foo *foo_init() {
    struct foo *f = malloc(sizeof(struct foo));
    foo_set_error_handler(f, default_error_handler);
    return f;
 }


 struct foo *f = foo_init();
 foo_something();

One interesting benefit of a callback is that it can be invoked multiple times, or none at all in the absence of errors in which there is no overhead on the happy path.

There is, however, an inversion of control. The calling code does not know if the callback was invoked. As such, it may make sense to also use an indicator.

There's a nice set of slides from CMU's CERT with recommendations for when to use each of the common C (and C++) error handling techniques. One of the best slides is this decision tree:

I would personally change two things about this flowcart.

First, I would clarify that sometimes objects should use return values to indicate errors. If a function only extracts data from an object but doesn't mutate the object, then the integrity of the object itself is not at risk and indicating errors using a return value is more appropriate.

Second, it's not always appropriate to use exceptions in C++. Exceptions are good because they can reduce the amount of source code devoted to error handling, they mostly don't affect function signatures, and they're very flexible in what data they can pass up the callstack. On the other hand, exceptions might not be the right choice for a few reasons:

1. C++ exceptions have very particular semantics. If you don't want those semantics, then C++ exceptions are a bad choice. An exception must be dealt with immediately after being thrown and the design favors the case where an error will need to unwind the callstack a few levels.

2. C++ functions that throw exceptions can't later be wrapped to not throw exceptions, at least not without paying the full cost of exceptions anyway. Functions that return error codes can be wrapped to throw C++ exceptions, making them more flexible. C++'s new gets this right by providing a non-throwing variant.

3. C++ exceptions are relatively expensive but this downside is mostly overblown for programs making sensible use of exceptions. A program simply shouldn't throw exceptions on a codepath where performance is a concern. It doesn't really matter how fast your program can report an error and exit.

4. Sometimes C++ exceptions are not available. Either they're literally not available in one's C++ implementation, or one's code guidelines ban them.

Since the original question was about a multithreaded context, I think the local error indicator technique (what's described in SirDarius's answer) was underappreciated in the original answers. It's threadsafe, doesn't force the error to be immediately dealt with by the caller, and can bundle arbitrary data describing the error. The downside is that it must be held by an object (or I suppose somehow associated externally) and is arguably easier to ignore than a return code.

Second approach lets the compiler produce more optimized code, because when address of a variable is passed to a function, the compiler cannot keep its value in register(s) during subsequent calls to other functions. The completion code usually is used only once, just after the call, whereas "real" data returned from the call may be used more often

If you want your program to crash and not know the reason, then go ahead and trust the programmers and c basic error handling.

I think it's best to build in some kind of error reporting, call it debug mode, turn it off when your want best performance and turn it on when you want to debug a issue. Hopefully you can hit it again.

There will be bugs, the question is how do you want to spend your days and nights looking for them.

Use setjmp.

http://en.wikipedia.org/wiki/Setjmp.h

http://aszt.inf.elte.hu/~gsd/halado_cpp/ch02s03.html

http://www.di.unipi.it/~nids/docs/longjump_try_trow_catch.html

#include <setjmp.h>
#include <stdio.h>

jmp_buf x;

void f()
{
    longjmp(x,5); // throw 5;
}

int main()
{
    // output of this program is 5.

    int i = 0;

    if ( (i = setjmp(x)) == 0 )// try{
    {
        f();
    } // } --> end of try{
    else // catch(i){
    {
        switch( i )
        {
        case  1:
        case  2:
        default: fprintf( stdout, "error code = %d\n", i); break;
        }
    } // } --> end of catch(i){
    return 0;
}

#include <stdio.h>
#include <setjmp.h>

#define TRY do{ jmp_buf ex_buf__; if( !setjmp(ex_buf__) ){
#define CATCH } else {
#define ETRY } }while(0)
#define THROW longjmp(ex_buf__, 1)

int
main(int argc, char** argv)
{
   TRY
   {
      printf("In Try Statement\n");
      THROW;
      printf("I do not appear\n");
   }
   CATCH
   {
      printf("Got Exception!\n");
   }
   ETRY;

   return 0;
}

There's a nice set of slides from CMU's CERT with recommendations for when to use each of the common C (and C++) error handling techniques. One of the best slides is this decision tree:

I would personally change two things about this flowcart.

First, I would clarify that sometimes objects should use return values to indicate errors. If a function only extracts data from an object but doesn't mutate the object, then the integrity of the object itself is not at risk and indicating errors using a return value is more appropriate.

Second, it's not always appropriate to use exceptions in C++. Exceptions are good because they can reduce the amount of source code devoted to error handling, they mostly don't affect function signatures, and they're very flexible in what data they can pass up the callstack. On the other hand, exceptions might not be the right choice for a few reasons:

1. C++ exceptions have very particular semantics. If you don't want those semantics, then C++ exceptions are a bad choice. An exception must be dealt with immediately after being thrown and the design favors the case where an error will need to unwind the callstack a few levels.

2. C++ functions that throw exceptions can't later be wrapped to not throw exceptions, at least not without paying the full cost of exceptions anyway. Functions that return error codes can be wrapped to throw C++ exceptions, making them more flexible. C++'s new gets this right by providing a non-throwing variant.

3. C++ exceptions are relatively expensive but this downside is mostly overblown for programs making sensible use of exceptions. A program simply shouldn't throw exceptions on a codepath where performance is a concern. It doesn't really matter how fast your program can report an error and exit.

4. Sometimes C++ exceptions are not available. Either they're literally not available in one's C++ implementation, or one's code guidelines ban them.

Since the original question was about a multithreaded context, I think the local error indicator technique (what's described in SirDarius's answer) was underappreciated in the original answers. It's threadsafe, doesn't force the error to be immediately dealt with by the caller, and can bundle arbitrary data describing the error. The downside is that it must be held by an object (or I suppose somehow associated externally) and is arguably easier to ignore than a return code.

The UNIX approach is most similar to your second suggestion. Return either the result or a single "it went wrong" value. For instance, open will return the file descriptor on success or -1 on failure. On failure it also sets errno, an external global integer to indicate which failure occurred.

For what it's worth, Cocoa has also been adopting a similar approach. A number of methods return BOOL, and take an NSError ** parameter, so that on failure they set the error and return NO. Then the error handling looks like:

NSError *error = nil;
if ([myThing doThingError: &error] == NO)
{
  // error handling
}

which is somewhere between your two options :-).

When I write programs, during initialization, I usually spin off a thread for error handling, and initialize a special structure for errors, including a lock. Then, when I detect an error, through return values, I enter in the info from the exception into the structure and send a SIGIO to the exception handling thread, then see if I can't continue execution. If I can't, I send a SIGURG to the exception thread, which stops the program gracefully.

I was pondering this issue recently as well, and wrote up some macros for C that simulate try-catch-finally semantics using purely local return values. Hope you find it useful.

I ran into this Q&A a number of times, and wanted to contribute a more comprehensive answer. I think the best way to think about this is how to return errors to caller, and what you return.

How

There are 3 ways to return information from a function:

1. Return Value
2. Out Argument(s)
3. Out of Band, that includes non-local goto (setjmp/longjmp), file or global scoped variables, file system etc.

Return Value

You can only return value is a single object, however, it can be an arbitrary complex. Here is an example of an error returning function:

  enum error hold_my_beer();

One benefit of return values is that it allows chaining of calls for less intrusive error handling:

  !hold_my_beer() &&
  !hold_my_cigarette() &&
  !hold_my_pants() ||
  abort();

This not just about readability, but may also allow processing an array of such function pointers in a uniform way.

Out Argument(s)

You can return more via more than one object via arguments, but best practice does suggest to keep the total number of arguments low (say, <=4):

void look_ma(enum error *e, char *what_broke);

enum error e;
look_ma(e);
if(e == FURNITURE) {
  reorder(what_broke);
} else if(e == SELF) {
  tell_doctor(what_broke);
}

Out of Band

With setjmp() you define a place and how you want to handle an int value, and you transfer control to that location via a longjmp(). See Practical usage of setjmp and longjmp in C.

What

1. Indicator
2. Code
3. Object
4. Callback

Indicator

An error indicator only tells you that there is a problem but nothing about the nature of said problem:

struct foo *f = foo_init();
if(!f) {
  /// handle the absence of foo
}

This is the least powerful way for a function to communicate error state, however, perfect if caller cannot respond to the error in a graduated manner anyways.

Code

An error code tells the caller about the nature of the problem, and may allow for a suitable response (from the above). It can be return value, or like the look_ma() example above an error argument.

Object

With an error object, the caller can be informed about arbitrary complicated issues. For example, an error code and a suitable human readable message. It can also inform the caller that multiple things went wrong, or an error per item when processing a collection:

struct collection friends;
enum error *e = malloc(c.size * sizeof(enum error));
...
ask_for_favor(friends, reason);
for(int i = 0; i < c.size; i++) {
   if(reason[i] == NOT_FOUND) find(friends[i]);
}

Instead of pre-allocating the error array, you can also (re)allocate it dynamically as needed of course.

Callback

Callback is the most powerful way to handle errors, as you can tell the function what behavior you would like to see happen when something goes wrong. A callback argument can be added to each function, or if customization uis only required per instance of a struct like this:

 struct foo {
    ...
    void (error_handler)(char *);
 };

 void default_error_handler(char *message) { 
    assert(f);
    printf("%s", message);
 }

 void foo_set_error_handler(struct foo *f, void (*eh)(char *)) {
    assert(f);
    f->error_handler = eh;
 }

 struct foo *foo_init() {
    struct foo *f = malloc(sizeof(struct foo));
    foo_set_error_handler(f, default_error_handler);
    return f;
 }


 struct foo *f = foo_init();
 foo_something();

One interesting benefit of a callback is that it can be invoked multiple times, or none at all in the absence of errors in which there is no overhead on the happy path.

There is, however, an inversion of control. The calling code does not know if the callback was invoked. As such, it may make sense to also use an indicator.

In addition to what has been said, prior to returning your error code, fire off an assert or similar diagnostic when an error is returned, as it will make tracing a lot easier. The way I do this is to have a customised assert that still gets compiled in at release but only gets fired when the software is in diagnostics mode, with an option to silently report to a log file or pause on screen.

I personally return error codes as negative integers with no_error as zero , but it does leave you with the possible following bug

if (MyFunc())
 DoSomething();

An alternative is have a failure always returned as zero, and use a LastError() function to provide details of the actual error.

Returning error code is the usual approach for error handling in C.

But recently we experimented with the outgoing error pointer approach as well.

It has some advantages over the return value approach:

• You can use the return value for more meaningful purposes.

• Having to write out that error parameter reminds you to handle the error or propagate it. (You never forget checking the return value of fclose, don't you?)

• If you use an error pointer, you can pass it down as you call functions. If any of the functions set it, the value won't get lost.

• By setting a data breakpoint on the error variable, you can catch where does the error occurred first. By setting a conditional breakpoint you can catch specific errors too.

• It makes it easier to automatize the check whether you handle all errors. The code convention may force you to call your error pointer as err and it must be the last argument. So the script can match the string err); then check if it's followed by if (*err. Actually in practice we made a macro called CER (check err return) and CEG (check err goto). So you don't need to type it out always when we just want to return on error, and can reduce the visual clutter.

Not all functions in our code has this outgoing parameter though. This outgoing parameter thing are used for cases where you would normally throw an exception.

I've used both approaches, and they both worked fine for me. Whichever one I use, I always try to apply this principle:

If the only possible errors are programmer errors, don't return an error code, use asserts inside the function.

An assertion that validates the inputs clearly communicates what the function expects, while too much error checking can obscure the program logic. Deciding what to do for all the various error cases can really complicate the design. Why figure out how functionX should handle a null pointer if you can instead insist that the programmer never pass one?

EDIT:If you need access only to the last error, and you don't work in multithreaded environment.

You can return only true/false (or some kind of #define if you work in C and don't support bool variables), and have a global Error buffer that will hold the last error:

int getObjectSize(MYAPIHandle h, int* returnedSize);
MYAPI_ERROR LastError;
MYAPI_ERROR* getLastError() {return LastError;};
#define FUNC_SUCCESS 1
#define FUNC_FAIL 0

if(getObjectSize(h, &size) != FUNC_SUCCESS ) {
    MYAPI_ERROR* error = getLastError();
    // error handling
}

The UNIX approach is most similar to your second suggestion. Return either the result or a single "it went wrong" value. For instance, open will return the file descriptor on success or -1 on failure. On failure it also sets errno, an external global integer to indicate which failure occurred.

For what it's worth, Cocoa has also been adopting a similar approach. A number of methods return BOOL, and take an NSError ** parameter, so that on failure they set the error and return NO. Then the error handling looks like:

NSError *error = nil;
if ([myThing doThingError: &error] == NO)
{
  // error handling
}

which is somewhere between your two options :-).

I personally prefer the former approach (returning an error indicator).

Where necessary the return result should just indicate that an error occurred, with another function being used to find out the exact error.

In your getSize() example I'd consider that sizes must always be zero or positive, so returning a negative result can indicate an error, much like UNIX system calls do.

I can't think of any library that I've used that goes for the latter approach with an error object passed in as a pointer. stdio, etc all go with a return value.

I was pondering this issue recently as well, and wrote up some macros for C that simulate try-catch-finally semantics using purely local return values. Hope you find it useful.

What you could do instead of returning your error, and thus forbidding you from returning data with your function, is using a wrapper for your return type:

typedef struct {
    enum {SUCCESS, ERROR} status;
    union {
        int errCode;
        MyType value;
    } ret;
} MyTypeWrapper;

Then, in the called function:

MyTypeWrapper MYAPIFunction(MYAPIHandle h) {
    MyTypeWrapper wrapper;
    // [...]
    // If there is an error somewhere:
    wrapper.status = ERROR;
    wrapper.ret.errCode = MY_ERROR_CODE;

    // Everything went well:
    wrapper.status = SUCCESS;
    wrapper.ret.value = myProcessedData;
    return wrapper;
} 

Please note that with the following method, the wrapper will have the size of MyType plus one byte (on most compilers), which is quite profitable; and you won't have to push another argument on the stack when you call your function (returnedSize or returnedError in both of the methods you presented).

I've used both approaches, and they both worked fine for me. Whichever one I use, I always try to apply this principle:

If the only possible errors are programmer errors, don't return an error code, use asserts inside the function.

An assertion that validates the inputs clearly communicates what the function expects, while too much error checking can obscure the program logic. Deciding what to do for all the various error cases can really complicate the design. Why figure out how functionX should handle a null pointer if you can instead insist that the programmer never pass one?

EDIT:If you need access only to the last error, and you don't work in multithreaded environment.

You can return only true/false (or some kind of #define if you work in C and don't support bool variables), and have a global Error buffer that will hold the last error:

int getObjectSize(MYAPIHandle h, int* returnedSize);
MYAPI_ERROR LastError;
MYAPI_ERROR* getLastError() {return LastError;};
#define FUNC_SUCCESS 1
#define FUNC_FAIL 0

if(getObjectSize(h, &size) != FUNC_SUCCESS ) {
    MYAPI_ERROR* error = getLastError();
    // error handling
}

I've used both approaches, and they both worked fine for me. Whichever one I use, I always try to apply this principle:

If the only possible errors are programmer errors, don't return an error code, use asserts inside the function.

An assertion that validates the inputs clearly communicates what the function expects, while too much error checking can obscure the program logic. Deciding what to do for all the various error cases can really complicate the design. Why figure out how functionX should handle a null pointer if you can instead insist that the programmer never pass one?

I definitely prefer the first solution :

int size;
if(getObjectSize(h, &size) != MYAPI_SUCCESS) {
  // Error handling
}

i would slightly modify it, to:

int size;
MYAPIError rc;

rc = getObjectSize(h, &size)
if ( rc != MYAPI_SUCCESS) {
  // Error handling
}

In additional i will never mix legitimate return value with error even if currently the scope of function allowing you to do so, you never know which way function implementation will go in the future.

And if we already talking about error handling i would suggest goto Error; as error handling code, unless some undo function can be called to handle error handling correctly.

EDIT:If you need access only to the last error, and you don't work in multithreaded environment.

You can return only true/false (or some kind of #define if you work in C and don't support bool variables), and have a global Error buffer that will hold the last error:

int getObjectSize(MYAPIHandle h, int* returnedSize);
MYAPI_ERROR LastError;
MYAPI_ERROR* getLastError() {return LastError;};
#define FUNC_SUCCESS 1
#define FUNC_FAIL 0

if(getObjectSize(h, &size) != FUNC_SUCCESS ) {
    MYAPI_ERROR* error = getLastError();
    // error handling
}

Second approach lets the compiler produce more optimized code, because when address of a variable is passed to a function, the compiler cannot keep its value in register(s) during subsequent calls to other functions. The completion code usually is used only once, just after the call, whereas "real" data returned from the call may be used more often

First approach is better IMHO:

• It's easier to write function that way. When you notice an error in the middle of the function you just return an error value. In second approach you need to assign error value to one of the parameters and then return something.... but what would you return - you don't have correct value and you don't return error value.
• it's more popular so it will be easier to understand, maintain

I definitely prefer the first solution :

int size;
if(getObjectSize(h, &size) != MYAPI_SUCCESS) {
  // Error handling
}

i would slightly modify it, to:

int size;
MYAPIError rc;

rc = getObjectSize(h, &size)
if ( rc != MYAPI_SUCCESS) {
  // Error handling
}

In additional i will never mix legitimate return value with error even if currently the scope of function allowing you to do so, you never know which way function implementation will go in the future.

And if we already talking about error handling i would suggest goto Error; as error handling code, unless some undo function can be called to handle error handling correctly.

I have done a lot of C programming in the past. And I really apreciated the error code return value. But is has several possible pitfalls:

• Duplicate error numbers, this can be solved with a global errors.h file.
• Forgetting to check the error code, this should be solved with a cluebat and long debugging hours. But in the end you will learn (or you will know that someone else will do the debugging).

The UNIX approach is most similar to your second suggestion. Return either the result or a single "it went wrong" value. For instance, open will return the file descriptor on success or -1 on failure. On failure it also sets errno, an external global integer to indicate which failure occurred.

For what it's worth, Cocoa has also been adopting a similar approach. A number of methods return BOOL, and take an NSError ** parameter, so that on failure they set the error and return NO. Then the error handling looks like:

NSError *error = nil;
if ([myThing doThingError: &error] == NO)
{
  // error handling
}

which is somewhere between your two options :-).

I have done a lot of C programming in the past. And I really apreciated the error code return value. But is has several possible pitfalls:

• Duplicate error numbers, this can be solved with a global errors.h file.
• Forgetting to check the error code, this should be solved with a cluebat and long debugging hours. But in the end you will learn (or you will know that someone else will do the debugging).

Here is an approach which I think is interesting, while requiring some discipline.

This assumes a handle-type variable is the instance on which operate all API functions.

The idea is that the struct behind the handle stores the previous error as a struct with necessary data (code, message...), and the user is provided with a function that returns a pointer to this error object. Each operation will update the pointed object so the user can check its status without even calling functions. As opposed to the errno pattern, the error code is not global, which make the approach thread-safe, as long as each handle is properly used.

Example:

MyHandle * h = MyApiCreateHandle();

/* first call checks for pointer nullity, since we cannot retrieve error code
   on a NULL pointer */
if (h == NULL)
     return 0; 

/* from here h is a valid handle */

/* get a pointer to the error struct that will be updated with each call */
MyApiError * err = MyApiGetError(h);

MyApiFileDescriptor * fd = MyApiOpenFile("/path/to/file.ext");

/* we want to know what can go wrong */
if (err->code != MyApi_ERROR_OK) {
    fprintf(stderr, "(%d) %s\n", err->code, err->message);
    MyApiDestroy(h);
    return 0;
}

MyApiRecord record;

/* here the API could refuse to execute the operation if the previous one
   yielded an error, and eventually close the file descriptor itself if
   the error is not recoverable */
MyApiReadFileRecord(h, &record, sizeof(record));

/* we want to know what can go wrong, here using a macro checking for failure */
if (MyApi_FAILED(err)) {
    fprintf(stderr, "(%d) %s\n", err->code, err->message);
    MyApiDestroy(h);
    return 0;
}

I definitely prefer the first solution :

int size;
if(getObjectSize(h, &size) != MYAPI_SUCCESS) {
  // Error handling
}

i would slightly modify it, to:

int size;
MYAPIError rc;

rc = getObjectSize(h, &size)
if ( rc != MYAPI_SUCCESS) {
  // Error handling
}

In additional i will never mix legitimate return value with error even if currently the scope of function allowing you to do so, you never know which way function implementation will go in the future.

And if we already talking about error handling i would suggest goto Error; as error handling code, unless some undo function can be called to handle error handling correctly.

First approach is better IMHO:

• It's easier to write function that way. When you notice an error in the middle of the function you just return an error value. In second approach you need to assign error value to one of the parameters and then return something.... but what would you return - you don't have correct value and you don't return error value.
• it's more popular so it will be easier to understand, maintain

What you could do instead of returning your error, and thus forbidding you from returning data with your function, is using a wrapper for your return type:

typedef struct {
    enum {SUCCESS, ERROR} status;
    union {
        int errCode;
        MyType value;
    } ret;
} MyTypeWrapper;

Then, in the called function:

MyTypeWrapper MYAPIFunction(MYAPIHandle h) {
    MyTypeWrapper wrapper;
    // [...]
    // If there is an error somewhere:
    wrapper.status = ERROR;
    wrapper.ret.errCode = MY_ERROR_CODE;

    // Everything went well:
    wrapper.status = SUCCESS;
    wrapper.ret.value = myProcessedData;
    return wrapper;
} 

Please note that with the following method, the wrapper will have the size of MyType plus one byte (on most compilers), which is quite profitable; and you won't have to push another argument on the stack when you call your function (returnedSize or returnedError in both of the methods you presented).

If you want your program to crash and not know the reason, then go ahead and trust the programmers and c basic error handling.

I think it's best to build in some kind of error reporting, call it debug mode, turn it off when your want best performance and turn it on when you want to debug a issue. Hopefully you can hit it again.

There will be bugs, the question is how do you want to spend your days and nights looking for them.

I have done a lot of C programming in the past. And I really apreciated the error code return value. But is has several possible pitfalls:

• Duplicate error numbers, this can be solved with a global errors.h file.
• Forgetting to check the error code, this should be solved with a cluebat and long debugging hours. But in the end you will learn (or you will know that someone else will do the debugging).

In addition the other great answers, I suggest that you try to separate the error flag and the error code in order to save one line on each call, i.e.:

if( !doit(a, b, c, &errcode) )
{   (* handle *)
    (* thine  *)
    (* error  *)
}

When you have lots of error-checking, this little simplification really helps.

Here is an approach which I think is interesting, while requiring some discipline.

This assumes a handle-type variable is the instance on which operate all API functions.

The idea is that the struct behind the handle stores the previous error as a struct with necessary data (code, message...), and the user is provided with a function that returns a pointer to this error object. Each operation will update the pointed object so the user can check its status without even calling functions. As opposed to the errno pattern, the error code is not global, which make the approach thread-safe, as long as each handle is properly used.

Example:

MyHandle * h = MyApiCreateHandle();

/* first call checks for pointer nullity, since we cannot retrieve error code
   on a NULL pointer */
if (h == NULL)
     return 0; 

/* from here h is a valid handle */

/* get a pointer to the error struct that will be updated with each call */
MyApiError * err = MyApiGetError(h);

MyApiFileDescriptor * fd = MyApiOpenFile("/path/to/file.ext");

/* we want to know what can go wrong */
if (err->code != MyApi_ERROR_OK) {
    fprintf(stderr, "(%d) %s\n", err->code, err->message);
    MyApiDestroy(h);
    return 0;
}

MyApiRecord record;

/* here the API could refuse to execute the operation if the previous one
   yielded an error, and eventually close the file descriptor itself if
   the error is not recoverable */
MyApiReadFileRecord(h, &record, sizeof(record));

/* we want to know what can go wrong, here using a macro checking for failure */
if (MyApi_FAILED(err)) {
    fprintf(stderr, "(%d) %s\n", err->code, err->message);
    MyApiDestroy(h);
    return 0;
}

Returning error code is the usual approach for error handling in C.

But recently we experimented with the outgoing error pointer approach as well.

It has some advantages over the return value approach:

• You can use the return value for more meaningful purposes.

• Having to write out that error parameter reminds you to handle the error or propagate it. (You never forget checking the return value of fclose, don't you?)

• If you use an error pointer, you can pass it down as you call functions. If any of the functions set it, the value won't get lost.

• By setting a data breakpoint on the error variable, you can catch where does the error occurred first. By setting a conditional breakpoint you can catch specific errors too.

• It makes it easier to automatize the check whether you handle all errors. The code convention may force you to call your error pointer as err and it must be the last argument. So the script can match the string err); then check if it's followed by if (*err. Actually in practice we made a macro called CER (check err return) and CEG (check err goto). So you don't need to type it out always when we just want to return on error, and can reduce the visual clutter.

Not all functions in our code has this outgoing parameter though. This outgoing parameter thing are used for cases where you would normally throw an exception.

I use the first approach whenever I create a library. There are several advantages of using a typedef'ed enum as a return code.

• If the function returns a more complicated output such as an array and it's length you do not need to create arbitrary structures to return.

  rc = func(..., int **return_array, size_t *array_length);
  

• It allows for simple, standardized error handling.

  if ((rc = func(...)) != API_SUCCESS) {
     /* Error Handling */
  }
  

• It allows for simple error handling in the library function.

  /* Check for valid arguments */
  if (NULL == return_array || NULL == array_length)
      return API_INVALID_ARGS;
  

• Using a typedef'ed enum also allows for the enum name to be visible in the debugger. This allows for easier debugging without the need to constantly consult a header file. Having a function to translate this enum into a string is helpful as well.

The most important issue regardless of approach used is to be consistent. This applies to function and argument naming, argument ordering and error handling.

I personally prefer the former approach (returning an error indicator).

Where necessary the return result should just indicate that an error occurred, with another function being used to find out the exact error.

In your getSize() example I'd consider that sizes must always be zero or positive, so returning a negative result can indicate an error, much like UNIX system calls do.

I can't think of any library that I've used that goes for the latter approach with an error object passed in as a pointer. stdio, etc all go with a return value.

Here's a simple program to demonstrate the first 2 bullets of Nils Pipenbrinck's answer here.

His first 2 bullets are:

• store all possible error-states in one typedef'ed enum and use it in your lib. Don't just return ints or even worse, mix ints or different enumerations with return-codes.

• provide a function that converts errors into something human readable. Can be simple. Just error-enum in, const char* out.

Assume you have written a module named mymodule. First, in mymodule.h, you define your enum-based error codes, and you write some error strings which correspond to these codes. Here I am using an array of C strings (char *), which only works well if your first enum-based error code has value 0, and you don't manipulate the numbers thereafter. If you do use error code numbers with gaps or other starting values, you'll simply have to change from using a mapped C-string array (as I do below) to using a function which uses a switch statement or if / else if statements to map from enum error codes to printable C strings (which I don't demonstrate). The choice is yours.

mymodule.h

/// @brief Error codes for library "mymodule"
typedef enum mymodule_error_e
{
    /// No error
    MYMODULE_ERROR_OK = 0,
    
    /// Invalid arguments (ex: NULL pointer where a valid pointer is required)
    MYMODULE_ERROR_INVARG,

    /// Out of memory (RAM)
    MYMODULE_ERROR_NOMEM,

    /// Make up your error codes as you see fit
    MYMODULE_ERROR_MYERROR, 

    // etc etc
    
    /// Total # of errors in this list (NOT AN ACTUAL ERROR CODE);
    /// NOTE: that for this to work, it assumes your first error code is value 0 and you let it naturally 
    /// increment from there, as is done above, without explicitly altering any error values above
    MYMODULE_ERROR_COUNT,
} mymodule_error_t;

// Array of strings to map enum error types to printable strings
// - see important NOTE above!
const char* const MYMODULE_ERROR_STRS[] = 
{
    "MYMODULE_ERROR_OK",
    "MYMODULE_ERROR_INVARG",
    "MYMODULE_ERROR_NOMEM",
    "MYMODULE_ERROR_MYERROR",
};

// To get a printable error string
const char* mymodule_error_str(mymodule_error_t err);

// Other functions in mymodule
mymodule_error_t mymodule_func1(void);
mymodule_error_t mymodule_func2(void);
mymodule_error_t mymodule_func3(void);

mymodule.c contains my mapping function to map from enum error codes to printable C strings:

mymodule.c

#include <stdio.h>

/// @brief      Function to get a printable string from an enum error type
/// @param[in]  err     a valid error code for this module
/// @return     A printable C string corresponding to the error code input above, or NULL if an invalid error code
///             was passed in
const char* mymodule_error_str(mymodule_error_t err)
{
    const char* err_str = NULL;

    // Ensure error codes are within the valid array index range
    if (err >= MYMODULE_ERROR_COUNT)
    {
        goto done;
    }

    err_str = MYMODULE_ERROR_STRS[err];

done:
    return err_str;
}

// Let's just make some empty dummy functions to return some errors; fill these in as appropriate for your 
// library module

mymodule_error_t mymodule_func1(void)
{
    return MYMODULE_ERROR_OK;
}

mymodule_error_t mymodule_func2(void)
{
    return MYMODULE_ERROR_INVARG;
}

mymodule_error_t mymodule_func3(void)
{
    return MYMODULE_ERROR_MYERROR;
}

main.c contains a test program to demonstrate calling some functions and printing some error codes from them:

main.c

#include <stdio.h>

int main()
{
    printf("Demonstration of enum-based error codes in C (or C++)\n");

    printf("err code from mymodule_func1() = %s\n", mymodule_error_str(mymodule_func1()));
    printf("err code from mymodule_func2() = %s\n", mymodule_error_str(mymodule_func2()));
    printf("err code from mymodule_func3() = %s\n", mymodule_error_str(mymodule_func3()));

    return 0;
}

Output:

Demonstration of enum-based error codes in C (or C++)
err code from mymodule_func1() = MYMODULE_ERROR_OK
err code from mymodule_func2() = MYMODULE_ERROR_INVARG
err code from mymodule_func3() = MYMODULE_ERROR_MYERROR

References:

You can run this code yourself here: https://onlinegdb.com/ByEbKLupS.

In addition the other great answers, I suggest that you try to separate the error flag and the error code in order to save one line on each call, i.e.:

if( !doit(a, b, c, &errcode) )
{   (* handle *)
    (* thine  *)
    (* error  *)
}

When you have lots of error-checking, this little simplification really helps.

The UNIX approach is most similar to your second suggestion. Return either the result or a single "it went wrong" value. For instance, open will return the file descriptor on success or -1 on failure. On failure it also sets errno, an external global integer to indicate which failure occurred.

For what it's worth, Cocoa has also been adopting a similar approach. A number of methods return BOOL, and take an NSError ** parameter, so that on failure they set the error and return NO. Then the error handling looks like:

NSError *error = nil;
if ([myThing doThingError: &error] == NO)
{
  // error handling
}

which is somewhere between your two options :-).

I have done a lot of C programming in the past. And I really apreciated the error code return value. But is has several possible pitfalls:

• Duplicate error numbers, this can be solved with a global errors.h file.
• Forgetting to check the error code, this should be solved with a cluebat and long debugging hours. But in the end you will learn (or you will know that someone else will do the debugging).

First approach is better IMHO:

• It's easier to write function that way. When you notice an error in the middle of the function you just return an error value. In second approach you need to assign error value to one of the parameters and then return something.... but what would you return - you don't have correct value and you don't return error value.
• it's more popular so it will be easier to understand, maintain

I use the first approach whenever I create a library. There are several advantages of using a typedef'ed enum as a return code.

• If the function returns a more complicated output such as an array and it's length you do not need to create arbitrary structures to return.

  rc = func(..., int **return_array, size_t *array_length);
  

• It allows for simple, standardized error handling.

  if ((rc = func(...)) != API_SUCCESS) {
     /* Error Handling */
  }
  

• It allows for simple error handling in the library function.

  /* Check for valid arguments */
  if (NULL == return_array || NULL == array_length)
      return API_INVALID_ARGS;
  

• Using a typedef'ed enum also allows for the enum name to be visible in the debugger. This allows for easier debugging without the need to constantly consult a header file. Having a function to translate this enum into a string is helpful as well.

The most important issue regardless of approach used is to be consistent. This applies to function and argument naming, argument ordering and error handling.

I've used both approaches, and they both worked fine for me. Whichever one I use, I always try to apply this principle:

If the only possible errors are programmer errors, don't return an error code, use asserts inside the function.

An assertion that validates the inputs clearly communicates what the function expects, while too much error checking can obscure the program logic. Deciding what to do for all the various error cases can really complicate the design. Why figure out how functionX should handle a null pointer if you can instead insist that the programmer never pass one?