Mailing List Archive

Asking for help in deciding what functions should be in the API
Tl;Dr Please scan the list below for functions, variables, and macros
that really XS writers have no business accessing directly.

There are about 200 functions and macros that Devel::PPPort thinks are
public, in the API, but for which there is no documentation. This is
somewhat more than perlapi lists as being public but not documented.

I view this as scandalous. We apparently have a bunch more things
implemented that could make contributors' lives easier, if only they
knew about them. Just yesterday I stumbled on delimcpy, undocumented.
I asked sisyphus earlier about several function that he was unaware of
and it turns that could have helped him solve a problem last year.

The bottom line is that it hurts us to have this sub-tribal knowledge
hidden away.

I suspect that Devel::PPPort is thinking more of these functions are API
than really are, due to flaws in it. And others in the list shouldn't
be API either. Just today I discovered that my_stat, while listed as
API, can't actually be being used, as it is simply a wrapper macro that
immediately calls a function not accessible outside the perl core; hence
will lead to a loader error if used.

I'm hoping those of you who are familiar with using XS and interfacing
with the API functions will scan the list below for things that really
only the core should be using. Like all the PL_foo variables. I bet
most of those should not be used directly outside the core, but rather
only though macros and functions that are public.

I've already cut this list down by dozens, either by documenting things
I know about or that I thought I could learn sufficiently, or by
deciding that they shouldn't be in the API. Afterwards is the list of
those. Correct me if I'm wrong.

Maybe the problem isn't as bad as it appears.

Things I'm particularly suspicious of are the PL_ variables, the things
that create OPs (I'm not sure that XS should be doing that), dumping
functions

amagic_call
amagic_deref_call
any_dup
atfork_lock
atfork_unlock
block_gimme
call_atexit
call_list
clear_defarray
clone_params_del
clone_params_new
deb
deb_nocontext
debop
debprofdump
debstack
debstackptrs
despatch_signals
dirp_dup
do_aspawn
do_close
do_join
do_open
do_openn
doref
do_spawn
do_spawn_nowait
dowantarray
dump_eval
dump_form
dump_mstats
dump_sub
dXSTARG
filter_del
fp_dup
get_context
get_mstats
get_op_descs
get_op_names
get_ppaddr
get_vtbl
gp_dup
gp_free
gp_ref
gv_add_by_type
Gv_AMupdate
gv_AVadd
gv_dump
gv_efullname3
gv_efullname4
gv_fullname3
gv_fullname4
gv_handler
gv_HVadd
gv_IOadd
gv_name_set
he_dup
hek_dup
hv_delayfree_ent
hv_eiter_p
hv_eiter_set
hv_free_ent
hv_ksplit
hv_name_set
hv_placeholders_get
hv_placeholders_set
hv_rand_set
hv_riter_p
hv_riter_set
init_stacks
init_tm
is_lvalue_sub
leave_scope
magic_dump
markstack_grow
mfree
mg_dup
mg_size
mro_get_from_name
mro_set_mro
my_chsize
my_cxt_init
my_dirfd
my_failure_exit
my_fflush_all
my_fork
my_pclose
my_popen
my_popen_list
my_socketpair
newANONATTRSUB
newANONHASH
newANONLIST
newANONSUB
newAVREF
newCVREF
newFORM
newGVgen
newGVgen_flags
newGVREF
newHVhv
newHVREF
newIO
newMYSUB
newPROG
newRV
new_stackinfo
newSVREF
op_refcnt_lock
op_refcnt_unlock
parser_dup
perl_alloc_using
perl_clone_using
PerlIO_context_layers
PerlIO_fill
PerlIO_unread
PL_compiling
PL_DBsignal
PL_debstash
PL_diehook
PL_dirty
PL_hints
PL_laststatval
PL_mess_sv
PL_no_modify
PL_perldb
PL_ppaddr
PL_signals
PL_stack_base
PL_stack_sp
PL_statcache
PL_stdingv
PL_sv_arenaroot
PL_tainted
PL_tainting
PL_Xpv
pmop_dump
pop_scope
pregfree
ptr_table_fetch
ptr_table_free
ptr_table_new
ptr_table_split
ptr_table_store
push_scope
re_compile
repeatcpy
rsignal_state
rvpv_dup
save_adelete
save_aelem
save_aelem_flags
save_alloc
save_generic_pvref
save_generic_svref
save_hdelete
save_helem
save_helem_flags
save_hints
save_op
save_padsv_and_mortalize
save_pushi32ptr
save_pushptr
save_pushptrptr
save_set_svflags
save_shared_pvref
savestack_grow
savestack_grow_cnt
save_vptr
scan_vstring
seed
set_context
share_hek
si_dup
ss_dup
start_subparse
sv_2pvutf8_flags
SV_CONST_RETURN
SV_COW_SHARED_HASH_KEYS
sv_dup
sv_dup_inc
sv_len_utf8_nomg
SV_MUTABLE_RETURN
sv_peek
sys_intern_clear
sys_intern_dup
sys_intern_init
unsharepvn
vdeb
XSprePUSH

Things I've removed from the above list; correct me if I made a mistake.
Some of these are because we have a macro (that should be documented
if it isn't already) that we prefer to be used instead. Such as CXINC
vs cxinc, I haven't listed most of those that I've changed in a branch
av_arylen_p
av_iter_p
cast_i32
cast_iv
cast_ulong
cast_uv
csighandler
csighandler1
csighandler3
cx_dump
cx_dup
do_gv_dump
do_gvgv_dump
do_hv_dump
do_op_dump
do_pmop_dump
do_sprintf
do_sv_dump
dounwind
dump_indent
dump_vindent
gv_check
hv_assert
hv_common
hv_common_key_len
moreswitches
newSVsv_flags
pad_setsv
perly_sighandler
pregfree2
regdump
regexec_flags
regfree_internal
reginitcolors
reg_named_buff_all
reg_named_buff_exists
reg_named_buff_fetch
reg_named_buff_firstkey
reg_named_buff_nextkey
reg_named_buff_scalar
regnext
runops_debug
runops_standard
stack_grow
str_to_version
sv_2pv
sys_init
sys_init3
Re: Asking for help in deciding what functions should be in the API [ In reply to ]
On Sun, Jul 26, 2020 at 09:09:43PM -0600, Karl Williamson wrote:
> Tl;Dr Please scan the list below for functions, variables, and macros that
> really XS writers have no business accessing directly.
>
> There are about 200 functions and macros that Devel::PPPort thinks are
> public, in the API, but for which there is no documentation. This is
> somewhat more than perlapi lists as being public but not documented.
>
> I view this as scandalous. We apparently have a bunch more things
> implemented that could make contributors' lives easier, if only they knew
> about them. Just yesterday I stumbled on delimcpy, undocumented. I asked
> sisyphus earlier about several function that he was unaware of and it turns
> that could have helped him solve a problem last year.
>
> The bottom line is that it hurts us to have this sub-tribal knowledge hidden
> away.
>
> I suspect that Devel::PPPort is thinking more of these functions are API
> than really are, due to flaws in it. And others in the list shouldn't be
> API either. Just today I discovered that my_stat, while listed as API,
> can't actually be being used, as it is simply a wrapper macro that
> immediately calls a function not accessible outside the perl core; hence
> will lead to a loader error if used.
>
> I'm hoping those of you who are familiar with using XS and interfacing with
> the API functions will scan the list below for things that really only the
> core should be using. Like all the PL_foo variables. I bet most of those
> should not be used directly outside the core, but rather only though macros
> and functions that are public.
>
> I've already cut this list down by dozens, either by documenting things I
> know about or that I thought I could learn sufficiently, or by deciding that
> they shouldn't be in the API. Afterwards is the list of those. Correct me
> if I'm wrong.
>
> Maybe the problem isn't as bad as it appears.
>
> Things I'm particularly suspicious of are the PL_ variables, the things that
> create OPs (I'm not sure that XS should be doing that), dumping functions
>
> amagic_call
> amagic_deref_call

Both should be API so XS code can resolve overloaded operators.

> any_dup

Used on CPAN and I think it's reasonable for use in XS.

> atfork_lock
> atfork_unlock

Needed so code that forks can lock correctly, and for calls to PTHREAD_ATFORK()

> block_gimme

API.

> call_atexit

API (so modules can register function to call when perl exits)

> call_list

API (it isn't essential, but it's useful utility)

> clear_defarray

API (but I don't expect to see much usage)

> clone_params_del
> clone_params_new

Both definitely API (see perl5140delta)

> deb
> deb_nocontext

Both definitely API

> debop
> debprofdump
> debstack
> debstackptrs

API, mostly for debugging.

> despatch_signals

This may need to be visible, but I don't think anything should be
directly calling it (including win32_async_check()), since we have
PL_signalhook.

That's about as far as I can go through this list for now.

I think this list needs to be worked on incrementally, both in
determining which are APIs and actual documentation.

Tony
Re: Asking for help in deciding what functions should be in the API [ In reply to ]
On Mon, Jul 27, 2020 at 03:57:59PM +1000, Tony Cook wrote:
> On Sun, Jul 26, 2020 at 09:09:43PM -0600, Karl Williamson wrote:
> > Tl;Dr Please scan the list below for functions, variables, and macros that
> > really XS writers have no business accessing directly.
> > amagic_call
> > amagic_deref_call
>
> > any_dup
>
> Used on CPAN and I think it's reasonable for use in XS.

But really its hooking into the internal details of how ithreads clones
an interpreter. XS code that uses it is being naughty IMHO.

> > block_gimme
>
> API.

I thought block_gimme() and Perl_gimme_V() were the internal
implementations of the API macros GIMME/GIMME_V?

> > clear_defarray
>
> API (but I don't expect to see much usage)

That's heavily tied to the semantics and implementation of @_,
an could well change when signatures ditch @_. It's prefer it not to be
API.


--
Please note that ash-trays are provided for the use of smokers,
whereas the floor is provided for the use of all patrons.
-- Bill Royston