[statnet_help] standardized coefficients in ERGMs

[Carter T. Butts]

Hi, Jing -

Robert's email pointed out some of the complexities here, but I think 
one might even more usefully turn the question around.  What do does one 
/want to mean/ by an effect size?  By a standardized coefficient?  Once 
that is decided, one can determine (1) if such a quantity is 
well-defined, and (2) if so, how it can be calculated.  As Robert 
observed, many ideas that analysts have about effect sizes and 
standardization are actually special properties of linear models, and do 
not generalize to the non-linear world.  Such ideas are of limited use 
in understanding even logistic regression, much less complex systems 
with "feedback" among the elements (which is what we ERGMs are 
representing).  But, on the other hand, there are lots of ways to talk 
about effects and relative effects that can be useful for comparative 
purposes.  For instance:

  - Trivially, the ERGM coefficients tell you about how the conditional 
log odds of an edge vary as a function of the covariates, and the rest 
of the graph.  This provides an effect that has a consistent meaning 
across networks, and indeed is about as comparable as anything gets in a 
nonlinear model.  (The meaning is /local/, in the sense that the change 
in log odds is taken relative to a particular state, but that's the 
price of living in a nonlinear world.)  For quantitative covariate 
effects, one can pick any scale one wants, so if you wanted to partially 
"standardize" by scaling the covariates to have unit variance, one could 
do this.  (I tend to think that this type of practice usually causes 
more problems than it solves in the long run, but there's nothing 
illegitimate about it.)

  - Via conditional simulation, one can evaluate e.g. the expected 
change in one or more target statistics, as a function of change in one 
(or a combination of) parameters.  This is local with respect to the 
parameter vector, and may or may not be useful to compare across graphs 
depending on one's choice of statistics, but it does capture (local) 
"net" effects due to feedback between statistics, etc.  If one has a 
concrete substantive question relating to some aspect of network 
structure, computing such effects may be insightful.  One can, further, 
partially "standardize" these effects by scaling them by the standard 
deviation of the target statistics (or a function thereof) at the base 
parameter vector.  This gives you the expected change in statistics per 
unit change in parameters (local to the current base value), in units of 
the standard deviation in those statistics.  As with the above case, 
whether this is a good idea depends on what you want to know, but it can 
be helpful in giving you a sense of the extent to which small changes in 
the parameters are making a large difference in network structure, 
relative to the variation that you would naturally expect to see in that 
structure.

  - I'm a fan these days of scenario evaluations, and things like 
virtual "knock-out" experiments: in the latter case, we compare the 
expectation of some target statistic (or some other distributional 
statistic) for particular parameters (e.g., the MLE of a fitted model) 
with what we get if one or more terms in the model are set to zero.  
That is, we reach in and "turn off" the term, and see what it does to 
the graph (as is done physically in a knock-out experiment, where one 
might e.g., "turn off" expression of some gene in a mouse and see what 
effect it has). This can be useful as a probe to better understand how 
particular mechanisms are contributing to the overall behavior of the 
model, and can be used to construct a certain type of "effect size" 
based on how the target changes when specific effects are removed. 
(Whether that is useful depends on what you want to know, of course, but 
it can be insightful.)  We can of course do knock-down/knock-in/knock-up 
versions, as well, as well as versions where we modify e.g. covariates 
rather than parameters - all boil down to trying to understand the 
implications of model terms on substantive behavior by comparing across 
hypothetical scenarios (whether of empirically plausible or entirely 
conceptual nature).

  - In some physical settings, the ERGM parameters have a pretty 
concrete meaning as effective forces: setting aside things like 
contributions from the reference measure, a given parameter is the 
energy per unit change in the statistic, times -1/(kT) where T is the 
system temperature and k is Boltzmann's constant.  So, what we see is 
the (additive inverse of the) energetic cost of changing a given 
statistic by one unit, relative to the size of typical energy 
fluctuations (i.e., kT).  Admittedly, this is not as immediately helpful 
for social networks, but there are other settings where T is known, and 
the adjusted parameters then have a very direct and absolute 
interpretation.  From this vantage point, our usual coefficients are 
already standardized, in the sense that they reflect (to gloss it a bit) 
costs of changing the graph relative to available resources.  I think 
this can be pushed a bit further even in the social case, but I think it 
is safe to say that this is still something that is being worked out.  
We'll have to see where it leads.

Anyway, those are just a few of the examples of things that folks are 
doing in this area.  I agree with Robert that we're not going to have 
some simple, generic recipe for how to think about effects that is ideal 
in all cases, but that doesn't exist even for linear models.  By turns, 
we have quite a lot of powerful ways to use and interpret the 
coefficients that we have, and I think that folks will continue to come 
up with new ones as the number of applications grows.  What needs to be 
in the driver's seat, in my view, are the substantive questions.  If 
folks know and can clearly articulate what they are trying to learn, 
then they are likely to be able to come up with ways to measure the 
right quantities.

Hope that is helpful,

-Carter

On 6/16/22 4:43 AM, Jing Chen wrote:

>

> Dear Statnet community,

>

> I am posting this question again – not sure if the previous one was 

> sent successfully.

>

> I am trying to compare the strength of an edge effect across ERGMs 

> (the identical model specification, but the outcome objects are 

> different). I am wondering how do we talk about effect sizes in the 

> ERGM world? Are the “regression” coefficients presented in the ERGM 

> output standardized? If not, is there a way to do so?

>

> Any information would be appreciated. Thank you!

>

> Jing Chen, Ph.D.

>

> Assistant professor

>

> Shanghai Jiao Tong University

>

>

> _______________________________________________

> statnet_help mailing list

> statnet_help at u.washington.edu

> https://urldefense.com/v3/__http://mailman13.u.washington.edu/mailman/listinfo/statnet_help__;!!CzAuKJ42GuquVTTmVmPViYEvSg!M_zeRwLHiLMJLk_Nxn_dg32pF8ECLVLcU2Sed2IKAaJFtBJa4hT_afhwHhFydSu0iOdUHi2jERtRUQqE9FnFwOxq$  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/statnet_help/attachments/20220616/39478cc8/attachment.html>