Algorithms that jointly allocate resources across different layers are envisioned to boost the performance of wireless systems. Recent results have revealed that two of the most important parameters that critically affect the resulting cross-layer designs are channel- and queue-state information (QSI). Motivated by these results, this paper relies on stochastic convex optimization to develop optimal algorithms that use instantaneous fading and queue length information to allocate resources at the transport (flow-control), link, and physical layers. Focus is placed on a cellular system, where an access point exchanges information with different users over flat-fading orthogonal channels. Both uplink and downlink setups are considered. The allocation strategies are obtained as the solution of a constrained utility maximization problem that involves average performance metrics. It turns out that the optimal allocation at a given instant depends on the instantaneous channel-state information (CSI) and Lagrange multipliers, which are associated with the quality-of-service (QoS) requirements and the operating conditions of the system. The multipliers are estimated online using stochastic approximation tools and are linked with the window-averaged length of the queues. Capitalizing on those links, queue stability and average queuing delay of the developed algorithms are characterized, and a simple mechanism is devised to effect delay priorities among users.