How to do software architecture evaluation ?

I've gotten an assignment doing a software architecture evaluation without getting access to the code. The customer have chosen a platform and wants to know how the major software vendor that they are using, complies to this platform. That is - how can my customer utilize the software vendor's solution on the chosen J2EE platform.

I searched a bit on the internet and found a few resources Carnegie Mellon Software Engineering Institute. They have a sample chapter from the book Evaluation Software Architectures. They talk about doing a SAAM analysis (Software Architecture Analysis Method), but I'm a bit unsure how this will work with a software vendor who is a bit protectionist.

Since services seems to be THE hype word these days, I was thinking about a services focus on the architectural review. I'm going to try to stress the vendor by asking questions about:

- Deployment diagram
- Documentation of services (use case it supports, data input and output documentation and state changes as a result of the service)
- Backward compatibility policy (does it exist, is it predictable)
- Business separation from client/channel
- Layering within the solution
- Available tests
- Other reuse points (stored procedures, portlets++)

I'm also going to prepare a few scenarios which is relevant for my customer and try to evaluate the answers I get.

In the end the customer wants a matrix where we can do a cap analysis between where we want the vendor to be and where they actually are. This isn't going to be easy...

Do you have any experience in this area? If so - write down your thoughts.