Separate concerns

Your class has too many responsibilities. It is:

1. Being a container like std::vector.
2. Doing CUDA memory management.
3. Adding row/column vector product functionality.

It might look like reasonable code now, but what if you add support for more mathematical operations? What if you want to support OpenCL? What if you want it to work with doubles instead of floats? It's best to separate concerns to keep everything managable.

First, the standard library already has a variable-length vector class: std::vector. It would be great if we could reuse that. By default, it uses the default memory allocator under the hood to allocate memory. However, you want to use CUDA functions to allocate memory. Luckily, you can provide a custom memory allocator to most STL contains:

template<typename T>
class CudaAllocator {
public:
    using value_type = T;

    T* allocate(std::size_t n) {
        T* result;
        cudaMallocManaged(&result, n * sizeof(T));
        return result;
    }

    void deallocate(T* p, std::size_t) {
        cudaFree(p);
    }
};

Now you can create a vector with memory managed by CUDA like so:

std::vector<float, CudaAllocator<float>> cudavec;

You can make a type alias for it to save some typing:

template<typename T = float>
using CudaVector = std::vector<T, CudaAllocator<T>>;

Instead of making the math functions member functions of a class, you can write out-of-class functions taking two parameters. For example:

template<typename T>
T dot_product(const CudaVector<T>& lhs, const CudaVector<T>& rhs)
{
    …
    if (lhs.size() != rhs.size()) {
        throw std::runtime_error("Error: Vectors must have the same length to perform Dot product");
    }

    T result{};

    for (std::size_t i = 0; i < lhs.size(); ++i) {
        result += lhs[i] * rhs[i];
    }

    return result;
}

The remaining issue is that you made a distinction between row and column vectors. You can create new types (not just aliases) for these:

tempate<typename T = float>
class CudaRowVector: public CudaVector<T> {
public:
    using CudaVector<T>::CudaVector; // expose the constructor of CudaVector<>
};

tempate<typename T = float>
class CudaColumnVector: public CudaVector<T> {
public:
    using CudaVector<T>::CudaVector;
};

And then the declaration of the dot product operation becomes:

template<typename T>
T dot_product(const CudaRowVector<T>& lhs, const CudaColumnVector<T>& rhs)
{
    …
}

This has the benefit that row/column errors are now caught at compile-time instead of at run-time.

Make use of standard library algorithms

The standard library comes with lots of algorithms that you can use, including std::inner_product() which you can use like so:

template<typename T>
T dot_product(const CudaRowVector<T>& lhs, const CudaColumnVector<T>& rhs)
{
    if (lhs.size() != rhs.size()) {
        throw std::runtime_error("Error: Vectors must have the same length to perform Dot product");
    }

    return std::inner_product(lhs.begin(), lhs.end(), rhs.begin(), T{});
}

I am sure there are CUDA equivalents of library functions for vector and matrix manipulations, so you could use those instead of writing your own kernels.