Edit: Hash versus Array
As cincodenada correctly pointed out in the comment, ysth gave a better answer: I should have answered your question with another question: "Do you really want to use a Perl array? A hash may be more appropriate."
An array allocates memory for all possible indices up to the largest used so-far. In your example, you allocate 24 cells (but use only 3). By contrast, a hash only allocates space for those fields that are actually used.
Array solution: scalar grep
Here are two possible solutions (see below for explanation):
print scalar(grep {defined $_} @a), "\n"; # prints 3
print scalar(grep $_, @a), "\n"; # prints 3
Explanation: After adding $a[23], your array really contains 24 elements --- but most of them are undefined (which also evaluates as false). You can count the number of defined elements (as done in the first solution) or the number of true elements (second solution).
What is the difference? If you set $a[10]=0, then the first solution will count it, but the second solution won't (because 0 is false but defined). If you set $a[3]=undef, none of the solutions will count it.
Hash solution (by yst)
As suggested by another solution, you can work with a hash and avoid all the problems:
$a{0} = 1;
$a{5} = 2;
$a{23} = 3;
print scalar(keys %a), "\n"; # prints 3
This solution counts zeros and undef values.