上篇博客分析到setUpHWComposer函数,这里我们继续分析图像合成的过程从doComposition函数开始,以及在这过程中解答一些上篇博客提出的疑问。
一、doComposition合成图层
doComposition这个函数就是合成所有层的图像
void SurfaceFlinger::doComposition() { ATRACE_CALL(); const bool repaintEverything = android_atomic_and(0, &mRepaintEverything); for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) { const sp<DisplayDevice>& hw(mDisplays[dpy]); if (hw->isDisplayOn()) { // transform the dirty region into this screen's coordinate space const Region dirtyRegion(hw->getDirtyRegion(repaintEverything)); // repaint the framebuffer (if needed) doDisplayComposition(hw, dirtyRegion); hw->dirtyRegion.clear(); hw->flip(hw->swapRegion); hw->swapRegion.clear(); } // inform the h/w that we're done compositing hw->compositionComplete(); } postFramebuffer(); }
上面函数遍历所有的DisplayDevice然后调用doDisplayComposition函数。然后我们再看看doDisplayComposition函数
void SurfaceFlinger::doDisplayComposition(const sp<const DisplayDevice>& hw, const Region& inDirtyRegion) { bool isHwcDisplay = hw->getHwcDisplayId() >= 0; if (!isHwcDisplay && inDirtyRegion.isEmpty()) { return; } Region dirtyRegion(inDirtyRegion); //swapRegion设置为需要更新的区域 hw->swapRegion.orSelf(dirtyRegion); uint32_t flags = hw->getFlags();//获得显示设备支持的更新方式标志 if (flags & DisplayDevice::SWAP_RECTANGLE) {//支持矩阵更新 dirtyRegion.set(hw->swapRegion.bounds()); } else { if (flags & DisplayDevice::PARTIAL_UPDATES) {//支持部分更新 dirtyRegion.set(hw->swapRegion.bounds()); } else { //将更新区域调整为整个窗口大小 dirtyRegion.set(hw->bounds()); hw->swapRegion = dirtyRegion; } } if (CC_LIKELY(!mDaltonize && !mHasColorMatrix)) { if (!doComposeSurfaces(hw, dirtyRegion)) return;//合成 } else { RenderEngine& engine(getRenderEngine()); mat4 colorMatrix = mColorMatrix; if (mDaltonize) { colorMatrix = colorMatrix * mDaltonizer(); } mat4 oldMatrix = engine.setupColorTransform(colorMatrix); doComposeSurfaces(hw, dirtyRegion);//合成 engine.setupColorTransform(oldMatrix); } // update the swap region and clear the dirty region hw->swapRegion.orSelf(dirtyRegion); // swap buffers (presentation) hw->swapBuffers(getHwComposer());//使用egl将egl中的合成好的图像,输出到DisplayDevice的mSurface中 }
这个函数设置下需要更新的区域,后面调用doComposeSurfaces函数来合成图层,调用完doComposeSurfaces函数后,如果需要egl合成图像话,在这个函数中合成好。而最后调用swapBuffers只是将egl合成好的图像输出到DisplayDevice的mSurface中。
我们再来看看doComposeSurfaces函数,我们先来看一开始的代码,先判断是否有egl合成,然后再看是否有hwc合成(硬件合成)
bool SurfaceFlinger::doComposeSurfaces(const sp<const DisplayDevice>& hw, const Region& dirty) { RenderEngine& engine(getRenderEngine()); const int32_t id = hw->getHwcDisplayId(); HWComposer& hwc(getHwComposer()); HWComposer::LayerListIterator cur = hwc.begin(id); const HWComposer::LayerListIterator end = hwc.end(id); bool hasGlesComposition = hwc.hasGlesComposition(id); if (hasGlesComposition) {//是否有egl合成 if (!hw->makeCurrent(mEGLDisplay, mEGLContext)) { ALOGW("DisplayDevice::makeCurrent failed. Aborting surface composition for display %s", hw->getDisplayName().string()); eglMakeCurrent(mEGLDisplay, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); if(!getDefaultDisplayDevice()->makeCurrent(mEGLDisplay, mEGLContext)) { ALOGE("DisplayDevice::makeCurrent on default display failed. Aborting."); } return false; } // Never touch the framebuffer if we don't have any framebuffer layers const bool hasHwcComposition = hwc.hasHwcComposition(id); if (hasHwcComposition) {//是否有hwc合成 // when using overlays, we assume a fully transparent framebuffer // NOTE: we could reduce how much we need to clear, for instance // remove where there are opaque FB layers. however, on some // GPUs doing a "clean slate" clear might be more efficient. // We'll revisit later if needed. engine.clearWithColor(0, 0, 0, 0); } else { // we start with the whole screen area const Region bounds(hw->getBounds()); // we remove the scissor part // we're left with the letterbox region // (common case is that letterbox ends-up being empty) const Region letterbox(bounds.subtract(hw->getScissor())); // compute the area to clear Region region(hw->undefinedRegion.merge(letterbox)); // but limit it to the dirty region region.andSelf(dirty); // screen is already cleared here if (!region.isEmpty()) { // can happen with SurfaceView drawWormhole(hw, region); } } if (hw->getDisplayType() != DisplayDevice::DISPLAY_PRIMARY) { // just to be on the safe side, we don't set the // scissor on the main display. It should never be needed // anyways (though in theory it could since the API allows it). const Rect& bounds(hw->getBounds()); const Rect& scissor(hw->getScissor()); if (scissor != bounds) { // scissor doesn't match the screen's dimensions, so we // need to clear everything outside of it and enable // the GL scissor so we don't draw anything where we shouldn't // enable scissor for this frame const uint32_t height = hw->getHeight(); engine.setScissor(scissor.left, height - scissor.bottom, scissor.getWidth(), scissor.getHeight()); } } } ......
我们来看hasGlesComposition函数和hasHwcComposition函数,就是看其对应的DisplayData中是否有hasFbComp和hasOvComp。
bool HWComposer::hasGlesComposition(int32_t id) const { if (!mHwc || uint32_t(id)>31 || !mAllocatedDisplayIDs.hasBit(id)) return true; return mDisplayData[id].hasFbComp; }
bool HWComposer::hasHwcComposition(int32_t id) const { if (!mHwc || uint32_t(id)>31 || !mAllocatedDisplayIDs.hasBit(id)) return false; return mDisplayData[id].hasOvComp; }而这两个值是在prepare中调用Hwc的prepare函数之后赋值的
status_t HWComposer::prepare() { ...... int err = mHwc->prepare(mHwc, mNumDisplays, mLists); ALOGE_IF(err, "HWComposer: prepare failed (%s)", strerror(-err)); if (err == NO_ERROR) { // here we're just making sure that "skip" layers are set // to HWC_FRAMEBUFFER and we're also counting how many layers // we have of each type. // // If there are no window layers, we treat the display has having FB // composition, because SurfaceFlinger will use GLES to draw the // wormhole region. for (size_t i=0 ; i<mNumDisplays ; i++) { DisplayData& disp(mDisplayData[i]); disp.hasFbComp = false; disp.hasOvComp = false; if (disp.list) { for (size_t i=0 ; i<disp.list->numHwLayers ; i++) { hwc_layer_1_t& l = disp.list->hwLayers[i]; //ALOGD("prepare: %d, type=%d, handle=%p", // i, l.compositionType, l.handle); if (l.flags & HWC_SKIP_LAYER) { l.compositionType = HWC_FRAMEBUFFER; } if (l.compositionType == HWC_FRAMEBUFFER) { disp.hasFbComp = true;//只要有一个layer是HWC_FRAMEBUFFER } if (l.compositionType == HWC_OVERLAY) { disp.hasOvComp = true;//有一个layer是HWC_OVERLAY } if (l.compositionType == HWC_CURSOR_OVERLAY) { disp.hasOvComp = true;//有一个layer是HWC_CURSOR_OVERLAY } } if (disp.list->numHwLayers == (disp.framebufferTarget ? 1 : 0)) {//layer的数量 有framebufferTarget为1 没有为0 disp.hasFbComp = true; } } else { disp.hasFbComp = true;//没有list } } } return (status_t)err; }
我们继续看doComposeSurfaces函数,下面这个函数当cur!=end代表起码有两个以上图层,然后遍历图层,当layer是HWC_FRAMEBUFFER代表是需要egl合成的,而HWC_FRAMEBUFFER_TARGET是egl合成后使用的直接就跳了,HWC_CURSOR_OVERLAY和HWC_OVERLAY是用HWC模块(硬件合成)的,也就不用调用Layer的draw方法。而如果图层只要1个或者没有,那么直接使用egl合成。
HWComposer::LayerListIterator cur = hwc.begin(id); const HWComposer::LayerListIterator end = hwc.end(id); ...... const Vector< sp<Layer> >& layers(hw->getVisibleLayersSortedByZ()); const size_t count = layers.size(); const Transform& tr = hw->getTransform(); if (cur != end) { //代表起码有两个以上图层 // we're using h/w composer for (size_t i=0 ; i<count && cur!=end ; ++i, ++cur) {//遍历图层 const sp<Layer>& layer(layers[i]); const Region clip(dirty.intersect(tr.transform(layer->visibleRegion))); if (!clip.isEmpty()) { switch (cur->getCompositionType()) { case HWC_CURSOR_OVERLAY: case HWC_OVERLAY: { const Layer::State& state(layer->getDrawingState()); if ((cur->getHints() & HWC_HINT_CLEAR_FB) && i && layer->isOpaque(state) && (state.alpha == 0xFF) && hasGlesComposition) { // never clear the very first layer since we're // guaranteed the FB is already cleared layer->clearWithOpenGL(hw, clip); } break; } case HWC_FRAMEBUFFER: { layer->draw(hw, clip);//只有是HWC_FRAMEBUFFER才会调用Layer的draw合成 break; } case HWC_FRAMEBUFFER_TARGET: { // this should not happen as the iterator shouldn't // let us get there. ALOGW("HWC_FRAMEBUFFER_TARGET found in hwc list (index=%zu)", i); break; } } } layer->setAcquireFence(hw, *cur); } } else { // we're not using h/w composer for (size_t i=0 ; i<count ; ++i) {//只有一个或者没有图层 就直接使用Layer的draw合成 const sp<Layer>& layer(layers[i]); const Region clip(dirty.intersect( tr.transform(layer->visibleRegion))); if (!clip.isEmpty()) { layer->draw(hw, clip); } } } // disable scissor at the end of the frame engine.disableScissor(); return true; }
Layer的draw我们就不看了主要是使用egl合成纹理,但是有一点疑问,我们从来没有把layer中的mActiveBuffer放到egl中去,那么egl又是怎么合成各个layer的呢,我想肯定客户进程在绘制各个layer的时候,也是用egl绘制的,所有后面合成的时候egl有各个layer的buffer。
后面我们再来看下DisplayDevice::swapBuffers函数,是使用eglSwapBuffers来把egl合成的数据放到mSurface中去。
void DisplayDevice::swapBuffers(HWComposer& hwc) const { // We need to call eglSwapBuffers() if: // (1) we don't have a hardware composer, or // (2) we did GLES composition this frame, and either // (a) we have framebuffer target support (not present on legacy // devices, where HWComposer::commit() handles things); or // (b) this is a virtual display if (hwc.initCheck() != NO_ERROR || (hwc.hasGlesComposition(mHwcDisplayId) && (hwc.supportsFramebufferTarget() || mType >= DISPLAY_VIRTUAL))) { EGLBoolean success = eglSwapBuffers(mDisplay, mSurface); if (!success) { EGLint error = eglGetError(); if (error == EGL_CONTEXT_LOST || mType == DisplayDevice::DISPLAY_PRIMARY) { LOG_ALWAYS_FATAL("eglSwapBuffers(%p, %p) failed with 0x%08x", mDisplay, mSurface, error); } else { ALOGE("eglSwapBuffers(%p, %p) failed with 0x%08x", mDisplay, mSurface, error); } } } else if(hwc.supportsFramebufferTarget() || mType >= DISPLAY_VIRTUAL) { EGLBoolean success = eglSwapBuffersVIV(mDisplay, mSurface); if (!success) { EGLint error = eglGetError(); ALOGE("eglSwapBuffersVIV(%p, %p) failed with 0x%08x", mDisplay, mSurface, error); } } status_t result = mDisplaySurface->advanceFrame(); if (result != NO_ERROR) { ALOGE("[%s] failed pushing new frame to HWC: %d", mDisplayName.string(), result); } }
二、FramebufferSurface收到egl合成数据
之前分析DisplayDevice时候,还分析了FramebufferSurface,我们这里再来看下。
在SurfaceFlinger.cpp中的init函数,在创建DisplayDevice之前,我们先调用createBufferQueue来创建了一个buffer的生产者和消费者,然后把消费者放入了FramebufferSurface,生产者放入了DisplayDevice中。
sp<IGraphicBufferProducer> producer; sp<IGraphicBufferConsumer> consumer; BufferQueue::createBufferQueue(&producer, &consumer, new GraphicBufferAlloc()); sp<FramebufferSurface> fbs = new FramebufferSurface(*mHwc, i, consumer); int32_t hwcId = allocateHwcDisplayId(type); sp<DisplayDevice> hw = new DisplayDevice(this, type, hwcId, mHwc->getFormat(hwcId), isSecure, token, fbs, producer, mRenderEngine->getEGLConfig());
我们先来看生产者,下面是DisplayDevice的构造函数,生产者作为参数直接新建了一个Surface,然后把这个Surface作为参数调用eglCreateWindowSurface返回的就是mSurface,之前我们分析最后egl合成的数据时调用eglSwapBuffers并且把数据放到mSurface,这样最后肯定就到消费者(FramebufferSurface)去了。
mNativeWindow = new Surface(producer, false); ANativeWindow* const window = mNativeWindow.get(); /* * Create our display's surface */ EGLSurface surface; EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY); if (config == EGL_NO_CONFIG) { config = RenderEngine::chooseEglConfig(display, format); } surface = eglCreateWindowSurface(display, config, window, NULL);
最后到消费者那端的onFrameAvailable,也就是FramebufferSurface的onFrameAvailable中,我们现在来分析下这个过程,也就解答了一个onFrameAvailable的疑惑。
FramebufferSurface的父类是ConsumerBase类,我们来看其构造函数。先是构造了mConsumer,这里其实就是BufferQueueConsumer类,后面调用了其consumerConnect方法。
ConsumerBase::ConsumerBase(const sp<IGraphicBufferConsumer>& bufferQueue, bool controlledByApp) : mAbandoned(false), mConsumer(bufferQueue) { mName = String8::format("unnamed-%d-%d", getpid(), createProcessUniqueId()); wp<ConsumerListener> listener = static_cast<ConsumerListener*>(this); sp<IConsumerListener> proxy = new BufferQueue::ProxyConsumerListener(listener); status_t err = mConsumer->consumerConnect(proxy, controlledByApp); if (err != NO_ERROR) { CB_LOGE("ConsumerBase: error connecting to BufferQueue: %s (%d)", strerror(-err), err); } else { mConsumer->setConsumerName(mName); } }
我们来看下BufferQueueConsumer类的consumerConnect方法,就是调用了connect方法。
virtual status_t consumerConnect(const sp<IConsumerListener>& consumer, bool controlledByApp) { return connect(consumer, controlledByApp); }
这个方法中将mCore->mConsumerListener = consumerListener,这个mCore就是BufferQueueCore类。我们再从ConsumerBase的构造函数看这个consumerListener参数其实就是FrameBufferSurface对象本身。
status_t BufferQueueConsumer::connect( const sp<IConsumerListener>& consumerListener, bool controlledByApp) { ATRACE_CALL(); if (consumerListener == NULL) { BQ_LOGE("connect(C): consumerListener may not be NULL"); return BAD_VALUE; } BQ_LOGV("connect(C): controlledByApp=%s", controlledByApp ? "true" : "false"); Mutex::Autolock lock(mCore->mMutex); if (mCore->mIsAbandoned) { BQ_LOGE("connect(C): BufferQueue has been abandoned"); return NO_INIT; } mCore->mConsumerListener = consumerListener;//设置回调 mCore->mConsumerControlledByApp = controlledByApp; return NO_ERROR; }
我们再看BufferQueueProducer::queueBuffer函数,这个函数应该是生产者已经使用好buffer了,这个使用会调用如下代码这个listener就是BufferQueueCore的mConsumerListener,传输的数据时BufferItem。再传之前把BufferItem的mGraphicBuffer清了,因为消费者可以自己获取buffer,不用通过BufferItem传。
item.mGraphicBuffer.clear(); item.mSlot = BufferItem::INVALID_BUFFER_SLOT; // Call back without the main BufferQueue lock held, but with the callback // lock held so we can ensure that callbacks occur in order { Mutex::Autolock lock(mCallbackMutex); while (callbackTicket != mCurrentCallbackTicket) { mCallbackCondition.wait(mCallbackMutex); } if (frameAvailableListener != NULL) { frameAvailableListener->onFrameAvailable(item); } else if (frameReplacedListener != NULL) { frameReplacedListener->onFrameReplaced(item); } ++mCurrentCallbackTicket; mCallbackCondition.broadcast(); }
这样就要FramebufferSurface的onFrameAvailable函数中去了,我们来看下这个函数。
void FramebufferSurface::onFrameAvailable(const BufferItem& /* item */) { sp<GraphicBuffer> buf; sp<Fence> acquireFence; status_t err = nextBuffer(buf, acquireFence); if (err != NO_ERROR) { ALOGE("error latching nnext FramebufferSurface buffer: %s (%d)", strerror(-err), err); return; } err = mHwc.fbPost(mDisplayType, acquireFence, buf); if (err != NO_ERROR) { ALOGE("error posting framebuffer: %d", err); } }这个函数先用nextBuffer获取数据,然后调用了HWComposer的fbPost函数。我们先来看下nextBuffer函数,这个函数主要通过acquireBufferLocked获取BufferItem,其中的mBuf就是buffer了。
status_t FramebufferSurface::nextBuffer(sp<GraphicBuffer>& outBuffer, sp<Fence>& outFence) { Mutex::Autolock lock(mMutex); BufferItem item; status_t err = acquireBufferLocked(&item, 0); if (err == BufferQueue::NO_BUFFER_AVAILABLE) { outBuffer = mCurrentBuffer; return NO_ERROR; } else if (err != NO_ERROR) { ALOGE("error acquiring buffer: %s (%d)", strerror(-err), err); return err; } if (mCurrentBufferSlot != BufferQueue::INVALID_BUFFER_SLOT && item.mBuf != mCurrentBufferSlot) { // Release the previous buffer. err = releaseBufferLocked(mCurrentBufferSlot, mCurrentBuffer, EGL_NO_DISPLAY, EGL_NO_SYNC_KHR); if (err < NO_ERROR) { ALOGE("error releasing buffer: %s (%d)", strerror(-err), err); return err; } } mCurrentBufferSlot = item.mBuf; mCurrentBuffer = mSlots[mCurrentBufferSlot].mGraphicBuffer; outFence = item.mFence; outBuffer = mCurrentBuffer; return NO_ERROR; }而这个acquireBufferLocked还是用mConsumer的acquireBuffer来获取BufferItem。mConsumer就是BufferQueueConsumer类。
status_t ConsumerBase::acquireBufferLocked(BufferItem *item, nsecs_t presentWhen, uint64_t maxFrameNumber) { status_t err = mConsumer->acquireBuffer(item, presentWhen, maxFrameNumber); if (err != NO_ERROR) { return err; } if (item->mGraphicBuffer != NULL) { mSlots[item->mBuf].mGraphicBuffer = item->mGraphicBuffer; } mSlots[item->mBuf].mFrameNumber = item->mFrameNumber; mSlots[item->mBuf].mFence = item->mFence; CB_LOGV("acquireBufferLocked: -> slot=%d/%" PRIu64, item->mBuf, item->mFrameNumber); return OK; }
回到FramebufferSurface的onFrameAvailable中这样获取了buffer之后,调用了HWComposer的fbPost方法。
三、egl合成数据在HWComposer的处理
继上面调用fbPost方法,我们来看下,这里是调用了setFramebufferTarget方法。
int HWComposer::fbPost(int32_t id, const sp<Fence>& acquireFence, const sp<GraphicBuffer>& buffer) { if (mHwc && hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) { return setFramebufferTarget(id, acquireFence, buffer); } else { acquireFence->waitForever("HWComposer::fbPost"); return mFbDev->post(mFbDev, buffer->handle); } }
我们来看下setFramebufferTarget方法,这里就是把该设备的DisplayData数据中的framebufferTarget填充,主要是其handle数据,这里就是egl合成好的数据buffer。
也就是最终egl合成好的数据放在DisplayData的framebufferTarget变量的handle中。
status_t HWComposer::setFramebufferTarget(int32_t id, const sp<Fence>& acquireFence, const sp<GraphicBuffer>& buf) { if (uint32_t(id)>31 || !mAllocatedDisplayIDs.hasBit(id)) { return BAD_INDEX; } DisplayData& disp(mDisplayData[id]); if (!disp.framebufferTarget) { // this should never happen, but apparently eglCreateWindowSurface() // triggers a Surface::queueBuffer() on some // devices (!?) -- log and ignore. ALOGE("HWComposer: framebufferTarget is null"); return NO_ERROR; } int acquireFenceFd = -1; if (acquireFence->isValid()) { acquireFenceFd = acquireFence->dup(); } // ALOGD("fbPost: handle=%p, fence=%d", buf->handle, acquireFenceFd); disp.fbTargetHandle = buf->handle;//egl合成好的数据 disp.framebufferTarget->handle = disp.fbTargetHandle;//egl合成好的数据,最终是放在这里 disp.framebufferTarget->acquireFenceFd = acquireFenceFd; return NO_ERROR; }
四、硬件模块合成
这样就剩最后一步了,把不管是普通layer的数据,还是egl合成好的数据发送到硬件模块合成了,最后就到显示设备了。
继第一节分析的doComposition函数最后会调用postFramebuffer函数,我们再来分析下这个函数,这个函数主要是调用了HWComposer的commit函数。
void SurfaceFlinger::postFramebuffer() { ATRACE_CALL(); const nsecs_t now = systemTime(); mDebugInSwapBuffers = now; HWComposer& hwc(getHwComposer()); if (hwc.initCheck() == NO_ERROR) { if (!hwc.supportsFramebufferTarget()) { // EGL spec says: // "surface must be bound to the calling thread's current context, // for the current rendering API." getDefaultDisplayDevice()->makeCurrent(mEGLDisplay, mEGLContext); } hwc.commit(); } ......我们来看下HWComposer的commit函数,这个函数就是先设置了egl的那个设备的surface和display,然后处理虚拟设备的outbuf等,最后调用了硬件模块合成到显示设备上。
status_t HWComposer::commit() { int err = NO_ERROR; if (mHwc) { if (!hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) { // On version 1.0, the OpenGL ES target surface is communicated // by the (dpy, sur) fields and we are guaranteed to have only // a single display. mLists[0]->dpy = eglGetCurrentDisplay();//设置下egl相关变量 mLists[0]->sur = eglGetCurrentSurface(EGL_DRAW); } for (size_t i=VIRTUAL_DISPLAY_ID_BASE; i<mNumDisplays; i++) { DisplayData& disp(mDisplayData[i]); if (disp.outbufHandle) {//只有虚拟设备需要设置outbuf mLists[i]->outbuf = disp.outbufHandle; mLists[i]->outbufAcquireFenceFd = disp.outbufAcquireFence->dup(); } } err = mHwc->set(mHwc, mNumDisplays, mLists);//调用硬件模块合成 ......