Inconsistent `arena_matrix`/`varmat` interaction between `=` and `+=`

[andrjohns]

Opening this one as an issue since I'm not sure what the correct/intentional behaviour should be!

The arena_matrix <-> var_value<Eigen::MatrixXd> interaction currently handles inplace writes differently to assignment.

Where modifying an arena_matrix after creating a var_value will also update the value of the var_value if an inplace write is used, but not if an assignment is used:

#include <stan/math/rev.hpp>
#include <iostream>

int main() {
  Eigen::MatrixXd m = Eigen::MatrixXd::Constant(1, 1, 1);

  stan::arena_t<Eigen::MatrixXd> a(m);
  stan::math::var_value<Eigen::MatrixXd> v(a);  // shares a's memory
  std::cout << v.val() << "\n";

  a += m;  // writes in place, v sees the new values
  std::cout << v.val() << "\n";

  a = a + m;  // rebinds a, v keeps the old memory
  std::cout << v.val() << std::endl;
}

Returns:

$ ./arena 
1
2
2

[andrjohns]

Ahh it looks like this might cause incorrect grads:

fill:

#include <stan/math/rev.hpp>
#include <iostream>

int main() {
  using stan::math::var;
  Eigen::VectorXd m(3);
  m << 1, 2, 3;

  stan::math::var_value<Eigen::VectorXd> x(m);
  var s = stan::math::sum(stan::math::square(x.head(2)));  // view of x
  stan::math::fill(x, 5.0);
  s.grad();

  std::cout << "expected: 2 4 0\n";
  std::cout << "actual:   " << x.adj().transpose() << std::endl;
}

expected: 2 4 0
actual:   10 10  0

precomputed_gradients:

#include <stan/math/rev.hpp>
#include <iostream>

int main() {
  using stan::math::var;
  Eigen::VectorXd m(3);
  m << 1, 2, 3;
  Eigen::VectorXd g = Eigen::VectorXd::Constant(3, 10);

  stan::math::var_value<Eigen::VectorXd> x(m);
  var s = stan::math::sum(x.head(2));  // view of x
  var p = stan::math::precomputed_gradients(
      0.0, std::vector<var>{}, std::vector<double>{},
      std::forward_as_tuple(x), std::forward_as_tuple(g));
  var total = s + p;
  total.grad();

  std::cout << "expected: 11 11 10\n";
  std::cout << "actual:   " << x.adj().transpose() << std::endl;
}

expected: 11 11 10
actual:   10 10 10

[SteveBronder]

So for a += m; I think that is a side effect of not hard copying arena matrices when we see them. I think that is fine, but the fill example and precomputed_gradients are definitely places we need to hard copy.

I'm going to look into this more as I'm sure there are other places we need to hard copy as well.