If you are importing into a PO, it should default to the item's purchase unit of measure and if different from the standard unit of measure, then it uses the purchase unit of measure's conversion factor.  

If you are importing into a SO, then it should default to using the sales unit of measure.

Can you post the item's unit of measures and conversions (you can paste a screenshot)?

Can you clarify what kind of document you are trying to import?

Also, the order in which you set fields in the import job is important.  If you want to override the unit of measure, make sure it is being set before the quantity is being set.  You can reorder the fields on the data tab to make sure you get them in the correct order.