Odtnhj

Here is a miminum example of my problem, I have a list of numpy arrays that look like this:

a = np.zeros([4,3])
b = np.ones([5,3])
my_list = [a, b]

my_list
[array([[0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.]]), array([[1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.]])]

Arrays in the list could have variable length in one (and only one) of the dimension (here 4 or 5).

What I would like to have eventually is a numpy array that is of dimension (2, "Variable-size", 3). The output should look something like the following:

array([[[0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.]],
 [[1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.]]])

np.vstack() or np.concatenate() do not return me the shape I am looking for as they are producing an output that has a (9, 3) shape.

You can't have numpy array with the shape: (2, "Variable-size", 3), but you can concatenate two arrays with the shape ("Variable-size", 3) to (shape1[0] + shape2[0], 3).
As you wrote:

I am looking for as they are producing an output that has a (9, 3) shape

numpy.concatenate()

can be a solution to your problem:

np.concatenate((a,b))

Out:

array([[0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.],
 [0., 0., 0.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.],
 [1., 1., 1.]])

If you still want to maintain a variable 3rd dimension in numpy, the only way to do it with padding, here with zero padding:

import numpy as np

a = np.array([
 [50., 50., 50.],
 [50., 50., 50.],
 [50., 50., 50.],
 [50., 50., 50.],
 [0., 0., 0.] # zero padding 
])

b = np.array([
 [80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.]
])

new_matrix = [a, b]

print(new_matrix)

Out:

[array([[50., 50., 50.],
 [50., 50., 50.],
 [50., 50., 50.],
 [50., 50., 50.],
 [ 0., 0., 0.]]), array([[80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.],
 [80., 80., 80.]])]

This method is usable widely at a sort of image processing solutions.
With this trick you can leverage all the positive properties of matrix operations in numpy too, but you can keep your data shape relatively flexible.
As Willem mentioned above numpy only work with "rectangular" data and operations with variable dimensional matrices would be in the most situation ambiguous.

If you don't even want to use any of the mentioned solution, you have to choose list and numpy array combinations i.e. numpy arrays with any dimensions in a list.