How to write a program without causing a memory leak in XS

Perl
XS
here

I will explain how to write a program without causing a memory leak in XS.

Perl memory management is a reference counting method

First of all, as a basic knowledge, please be aware that Perl's memory management is performed by reference counting method. In the reference counting method, the memory is released when the reference count reaches 0. In other words, freeing memory in Perl is equivalent to setting the reference count to 0.

Let me explain a little more about the reference counting method. For Perl variable, the reference count for that variable is 1 when you first declare the variable. Also, when a reference to a variable is made, the reference count is incremented by 1.

{
  # The reference count for $str is 1.
  my $str = 'Hello';
  
  # The reference count for $str is 2. $str_ref reference count is 1
  my $str_ref = \$str;
}

Also, in Perl, variable are automatically released when you exit scope. This is because the variable is automatically in a state called mortal. The concept of mortal is very important. Mortal means "when the scope is removed, the reference count is automatically decremented by 1."

The above code describes what happens when you get out of scope.

{
  # The reference count for $str is 1.
  my $str = 'Hello';
  
  # The reference count for $str is 2. $str_ref reference count is 1
  my $str_ref = \$str;
}
# The reference count of $str_ref is decremented by 1 to 0.
# $str_ref is freed because the reference count of $str_ref is now 0.
# Now that $str_ref has been released, the reference count for $str has gone from 2 to 1.
# The reference count of $str is decremented by 1 to 0.
# $str is freed because the reference count of $str is 0.
# ('Hello' is contained inside $str, which is also released)

Following this process, the memory is released.

Basics of memory management in XS

Next, based on this, I will explain the basics of memory management in XS.

Perl variable

First, let's take a brief look at Perl variable.

I will explain Perl variable. First, internally, a scalar variable are represented by "SV * type". Arrays are represented by "AV * type" and hashes are represented by "HV * type". Let's remember this three first. Of course, the reference can be assigned to "SV * type". And, internally, keep in mind that "AV * type" and "HV * type" are derived from "SV * type". This means that you can upcast and downcast.

The following function is used to create a scalar variable. Keep in mind that in XS, the functions you create are different for strings, floats, and integers.

SV * sv_str = newSvPV ("Hello", 0);
SV * sv_num = newSvNV (1.2);
SV * sv_num_int = newSvIV (4);

The following functions are used to generate arrays and hashes.

AV * av_nums = newAV ();
HV * hv_scores = newHV ();

The reference is generated by the following function.

SV * sv_str_ref = newRV_inc (sv_str);

There is another function called newRV, but when generating a reference, it is a rule to increase the reference count by 1 with newRV_inc.

Make all created variable mortal

Then proceed to memory management. The ironclad rule of Perl's memory management is that all newly created Perl variable should be mortal. By making it mortal, the reference count of variable that are out of scope is decremented by 1 and the memory is automatically freed.

Make all newly created Perl variable mortal.

Use the sv_2mortal function to make it mortal. It takes an "SV * type" as an argument and the return value is a mortalized "SV * type".

sv_2mortal (SV * sv_var)

Use it as follows:

SV * sv_str = sv_2mortal (newSvPV ("Hello", 0));

In order to pass "AV * type", "HV * type", etc. to sv_2mortal, it is necessary to upcast to "SV * type" and downcast to "AV * type" "HV * type" when receiving further. ..

AV * sv_nums = (AV *) sv_2mortal ((SV *) newAV ());
HV * hv_nums = (AV *) sv_2mortal ((SV *) newHV ());

Also use sv_2mortal to create a reference.

SV * sv_str_ref = sv_2mortal (newRV_inc (sv_str));

Keeping the variable mortal in this way decrements the reference count by 1 and automatically releases it when Perl scope ends. Let's distinguish it because it is the end of Perl scope, not C scope. Internally, the entire XS function, such as the one below, is enclosed in Perl scope.

SV *
foo (...)
  PPCODE:
{
/* Starting Perl Scope */  
  SV * sv_str = sv_2mortal (newSvPV ("Hello", 0));
  
  XSRETURN (0);
  
/* End of Perl scope */}

When returned as a return value as shown below, when returning to the Perl code, the reference count is incremented by 1, and since it is mortal, the reference count is decremented by 1, so the reference count does not change as a result. ..

SV *
foo (...)
  PPCODE:
{
/* Starting Perl Scope */  
  SV * sv_str = sv_2mortal (newSvPV ("Hello", 0));
  
/* Put it on the stack and return it as a return value */  XPUSHs (sv_str);
  XSRETURN (1);
  
/* End of Perl scope */}

When storing data in arrays and hashes

If you code while observing the above iron rules, memory release will not work if you store data in arrays and hashes. This is because arrays and hashes decrement the reference count of the "SV * type" data they contain by one when they are destroyed by themselves.

To avoid this, you need to manually increment the reference count by 1 if you want to store the data in arrays and hashes. For arrays, this is the case when using av_push, av_store, and for hashes, using hv_store. Use the SvREFCNT_inc function to increase the reference count.

/ * When storing in an array */SV * sv_num = sv_2mortal (newSvIV (3));
AV * av_nums = (AV *) sv_2mortal ((SV *) newAV ());
av_push(av_nums, SvREFCNT_inc (sv_num));

/ * When storing in a hash */SV * sv_score_math = sv_2mortal (newSViv (60));
HV * hv_scores = (HV *) sv_2mortal ((SV *) newHV ());
hv_store (hv_scores, "math", strlen ("math"), SvREFCNT_inc (sv_score_math), 0);

When storing Perl data in a C structure

If you want to store Perl data in a C structure, you need to manage the memory yourself. As a premise of this story, you should look at How to treat C structs as Perl objects.

Let's consider a case where you want to save an "SV * type" in a C language structure. I tried to declare a structure called People with a member called sv_name.

struct People {
 SV * sv_name;
};

In this case, use SvREFCNT_inc to increase the reference count when substituting. Otherwise, the reference count will be decremented by 1 the moment you exit Perl's scope and will be released on its own.

SV *
foo (...)
  PPCODE:
{
  /* omit */
  
/* Create structure (create as pointer) */  People * people = (People *) malloc (sizeof (P)eople));
  SV * sv_name = sv_2mortal (newSvPV ("kimoto", 0));
  people->name = SvREFCNT_inc (sv_name);
  
  /* omit */
}

Then, in the destructor, use SvREFCNT_dec to lower the reference count by one.

void
DESTORY (...)
  PPCODE:
{
  // get the object
  SV * people_obj = ST (0);
  
  // Dereference
  SV * people_sv = SvROK (people_obj)? SvRV (people_obj): people_obj;
  
  // Convert SV * type to size_t type
  size_t people_iv = SvIV (people_sv);
  
  // Convert size_t type to pointer
  People * people = INT2PTR (People *, people_iv);
  
  // Release sv_name
  SvREFCNT_dec (people->sv_name);
  
  // Release People *
  free (people);
  
  XSRETURN (0);
}

If you want to store data in a member of the C world structure (same as the class) in this way, you need to manually increase or decrease the reference count.

Summary

In summary, there are three main points.

If you create a new Perl variable, use sv_2mortal to make the variable mortal.
When storing values in arrays and hashes, use SvREFCNT_inc to manually increase the reference count.
When saving to a member of a structure, use SvREFCNT_inc to increase the reference count when storing, and use SvREFCNT_dec to decrease the reference count by 1 in the destructor.

This should work in most cases.

Reference

The functions used in XS mentioned in this explanation are explained in detail in the following articles.

perlapi - API for data manipulation in XS

Perl ABC